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FOREWORD 


The  work  described  In  this  report  was  performed  by  the  author  as 
a  case  study  In  partial  fulfillment  of  a  Master  of  Science 
Engineering  Management  degree  at  the  University  of  Dayton.  Selection 
of  onboard  simulation  as  a  topic  for  study  was  motivated  by  the 
author's  work  experience  as  flight  test  director  for  the  USAF 
Integrated  Flight  and  Fire  Control  (IFFC)  advanced  development 
program.  The  author  extends  sincere  appreciation  to  Mrs  Marvelln 
Dotterer,  Ms  Shirley  Milne,  and  Ms  Colleen  Goebel  for  typing  support; 
Mr  Gary  Hellmann  (AFWAL/FIGX) ,  Lieutenant  Brian  Brady  (AFWAL/FIGX) 
and  Mr  Charles  Scolattl  (McDonnell  Aircraft  Co.)  for  consultation. 

This  case  study  consists  of  seven  chapters.  Chapter  1  Introduces 
the  subject  matter  with  a  statement  of  the  problem  to  be  addressed 
and  a  brief  preview  of  the  approach  to  be  applied  In  accomplishing 
this  study.  The  reader  should  develop  an  understanding  of  how  the 
United  States  Air  Force  (USAF)  trains  F— 15  pilots  to  use  the  aircraft 
gun  system  for  air-to-air  combat  and  some  of  the  limitations  of 
current  training  methods.  The  concept  of  inflight  Onboard  Simulation 
(OBS)  Is  introduced,  and  Its  potential  for  improved  training  is 
discussed.  Decision  theory  is  suggested  as  the  mechanism  for  explor¬ 
ing  alternative  applications  of  OBS. 

Chapter  2  provides  a  background  of  the  subject  matter  with  a  more 
detailed  description  of  OBS  and  decision  theory.  Original 


ill 


development  and  demonstration  of  OBS  is  presented  along  with 
discussion  of  strengths  and  weaknesses.  Future  applications  bring 
out  the  strong  potential  of  the  technology.  Decision  theory  is 
discussed  In  the  context  of  multiple  attribute  utility  analysis  used 
in  this  case  study  rather  then  a  broader  discussion  of  all  possible 
decision  methods.  Appropriate  examples  are  referenced. 

The  value  of  OBS  Is  developed  in  Chapter  3  by  studying  experience 
gained  in  actual  use.  An  estimate  of  dollars  saved  during  the 
Integrated  Flight  and  Fire  Control  program  is  computed.  Baseline 
performance  statistics  are  derived  for  use  in  later  chapters. 

A  basic  structure  for  the  decision  analysis  is  preaented  in 
Chapter  4.  This  includes  definition  of  objectives  for  applying  OBS 
to  pilot  training  and  developing  a  Hat  of  reaaonable  alternatives. 
Effectlveneas  measures,  called  attributes,  are  assigned  which  provide 
the  means  for  quantification  of  objectives. 

Chapter  5  deals  with  the  core  concept  of  utility  analysis  deci¬ 
sion  theory.  That  is,  a  decision  maker's  attitude  toward  the  value 
of  possible  outcomes.  This  might  also  bo  called  preferences  for  the 
outcomes.  Utility  functions  are  developr i  for  each  of  the  attributes 
assigned  in  the  previous  chapter.  A  matrix  of  specific  utility 
values  is  developed  by  assigning  a  value  to  all  attributes  for  each 
alternative. 

Chapter  6  presents  the  execution  of  the  multivariate  utility 
analysis.  Expected  utility  is  calculated  for  each  alternative  using 
the  general  form  of  the  utility  function.  A  crose-section  of  indivi- 


dual  decision  makers  Is  used  to  assign  scaling  factors  indicative  of 


each  individual's  attitude  of  the  relative  Importance  of  the  different 
attributes.  The  expected  utilities  are  recomputed  for  each  decision 
maker. 

Results  and  conclusions  are  presented  In  Chapter  7.  Recommenda¬ 
tions  are  based  on  the  fact  that  the  most  desirable  alternative 
exhibits  the  highest  expected  utility. 
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CHAPTER  1 


INTRODUCTION 


1.1  Problem  Statement 

Inflight  Onboard  Simulation  (OBS)  is  a  concept  which  allows  the 
pilot  of  a  tactical  fighter  aircraft  to  fly  realistic  weapon  delivery 
encounters  against  a  computer  generated  synthetic  target.  This 
target  is  generated  onboard  the  aircraft  by  a  dynamic  target  model 
and  is  presented  visually  to  the  pilot  on  his  head-up  display  (HUD). 
The  OBS  concept  was  developed  and  demonstrated  during  the  recently 
completed  United  States  Air  Force  Integrated  Flight  and  Fire  Control 
(IFFC)  advanced  development  test  program.  OBS  was  used  extensively 
during  the  IFFC  program  and  holds  significant  potential  for  applica¬ 
tion  to  operational  aircraft  as  a  pilot  training  device.  The  devised 
objectives  of  this  study  are  to  explore  the  attributes  of  OBS  and, 
through  decision  analysis,  develop  a  strategy  for  the  USAF  to  deter¬ 
mine  how  OBS  might  be  applied  to  operational  aircraft  for  effective 
pilot  training. 

For  a  long  time,  the  aircraft  gun  was  the  primary  weapon  for 
aerial  combat.  During  recent  years,  missiles  have  replaced  the  gun 
as  the  primary  weapon,  but  the  gun  remains  an  effective  and 
complementary  weapon.  "It  is  a  close-in  all-aspect  weapon",  and  due 
to  minimum  range  restrictions  for  employing  missiles,  "inside  3,000 
feet,  against  a  maneuvering  target  at  most  aspects,  the  gun  is  your 
only  weapon"  (4:9).  However,  effective  use  of  the  gun  is  highly 
dependent  upon  the  amount  and  quality  of  training  which  a  pilot 
receives. 
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Current  gun  training  consists  of  basically  two  types;  1) 
live-fire  attacks  against  atrial  tow  targets  and  2)  non-firing 
dynamic  encounters  against  manned  target  aircraft.  Each  type  of 
training  has  advantages  and  disadvantages  which  affect  the  quality  of 
training  in  representing  actual  combat  conditions. 

Live-fire  training  encounters  allow  pilots  the  opportunity  to 
actually  fire  the  aircraft  gun.  Inexpensive  plywood  targets  csllsd 
Darts  ars  deployed  on  a  4 >000  foot  cable  behind  an  F-4  aircraft  and 
the  attacking  aircraft  shoots  at  the  Dart.  It  is  certainly  desirable 
for  a  pilot  to  experience  the  sensation  of  firing  bullets  and  become 
familiar  with  the  switchology  and  procedures  for  firing  the  gun. 
Additional  advantages  are  that  the  gun  mechanism  is  checked  out  by 
periodic  firing,  and  boresight  of  the  gun  can  be  confirmed  if  the 
target  is  hit.  Major  disadvantages  are  created  because  the  engage¬ 
ment  dynamics  must  be  rigidly  controlled  to  prevent  sny  chance  of 
bullets  accldcntslly  hitting  the  manned  tow  aircraft.  Target  aspect 
sngle  is  usually  limited  to  conditions  shown  in  Figure  1-1  (20),  end 
the  tsrget  is  airspeed  limited.  The  net  result  is  that  live  fire 
encounters  against  tow  targets  do  not  present  the  dynamics  of  sn 
evasive  target  aircraft  that  wants  to  avoid  being  klllsd.  Another 
disadvantage  is  the  lack  of  an  effective  scoring  system  to  know  where 
the  bullets  are  going.  If  bulletB  go  through  the  Dart,  then  bullet 
position  is  known.  If  the  bullets  miss,  it  is  difficult  to  determine 
whether  the  pilot  did  not  track  the  target  properly,  or  if  the  gun  is 
out  of  boresight  (misaligned),  or  combinations  of  both. 
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FIGURE  1-1 

LIVE-FIRE  TRAINING  ENVELOPE 


FIGURE  1-2 

TACTICAL  TRAINING  ENVELOPE 


The  majority  of  gun  training  la  performed  in  a  tactical  mode 
where  a  maneuvering  manned  aircraft  la  uaed  as  a  target.  No  bullets 
are  fired  for  obvious  reasons,  but  very  realistic  dynamic  encounters 
are  created  when  the  target  aircraft  maneuvers  while  attempting  to 
escape.  Gun  camera  film  records  the  HUD  scene  when  the  attacking 
pilot  pulls  the  trigger.  Post-flight  Inspection  of  the  film  Is  used 
to  determine  If  the  taTget  should  have  been  hit.  There  are  some 
major  disadvantages  to  this  type  of  training.  Due  to  risk  of  mld-alr 
collision,  front  quarter  or  head-on  encounters  are  prohibited  (4). 
Additionally,  the  aircraft  must  never  come  within  1,000  feet  of  each 
other.  Figure  1-2  depicts  the  allowable  zones  for  training. 

Criteria  for  scoring  kills  from  gun  camera  film  Is  based  upon 
consecutive  frames  where  the  gunslght  plpper  Is  superimposed  on  the 
target  aircraft  (4).  This  assumption  Is  valid  for  rear  quarter  shots, 
but  when  target  aspect  angle  Is  greater  than  approximately  40  de¬ 
grees,  lead  angle  must  be  Inserted  in  order  to  hit  the  target  (4). 
There  Is  currently  no  convenient  way  to  estimate  proper  lead  angle 
from  gun  film.  Onboard  scoring  would  eliminate  this  scoring  problem 
because  It  would  provide  an  accurate  model  to  determine  hits, 

OBS  could  be  very  useful  toward  reducing  or  eliminating  some  of 
the  shortfalls  in  current  training  methods.  The  synthetic  airborne 
target  can  be  used  for  training  in  the  front  quarter  where  encounters 
are  head-on.  This  can  be  done  safely  without  risk  of  collision. 
Benefit  would  be  very  high  because  of  the  significant  number  of 
combat  encounters  which  would  be  expected  in  the  front  quarter  (19). 


Onboard  scoring  would  give  pilots  real-tins  feedback,  telling 
then  the  position  of  the  bullet  stream  and  an  estimate  of  the  nunber 
of  hits  scored.  When  pilots  know  where  they  missed,  they  can  compen¬ 
sate  on  the  next  shot.  This  learning  process  Is  much  more  productive 
than  looking  at  gun  camera  film  well  after  the  mission  and  attempting 
to  remember  what  happened.  If  a  gun  is  Improperly  boresighted, 
onboard  scoring  can  help  to  determine  such.  Whenever  scoring  shows 
hits  on  the  target,  but  no  hits  were  experienced,  one  ehould  euspect 
a  gun  alignment  problem. 

For  tactical  training,  scoring  would  eliminate  sole  dependence 
upon  plpper  on  target  as  scoring  criteria.  As  aspect  angle  increases 
beyond  40  degrees,  lead  angle  is  required  to  hit  the  target  which 
makes  the  plpper  on  target  criteria  incorrect.  The  pilot  can  quickly 
learn  by  trial  and  error  what  amount  of  lead  angle  is  needed  In  given 
situations  to  score  hits  on  target. 

OBS  Is  not  without  drawbacks,  however.  As  previously  mentioned, 
currently  available  versions  of  OBS  are  symbology  limited  to  the  HUD 
f ield-of-view.  Encounters  can  ba  structured  to  keep  the  target 
within  the  HUD  and  demonstrate  all  aspect  encounters.  This  teaches 
pilots  proper  lead  angles  to  achieve  bullets  on  target,  but  le  week 
in  teaching  air  combat  maneuvering  tactics.  That  is,  an  ability  to 
maneuver  relative  to  an  adversary  aircraft  and  close  In  for  a  shot 
while  guarding  against  a  maneuver  by  the  adversary  which  placea  him 
In  a  position  of  advantage.  A  helmet  mounted  display  which  allows 
all-aspsct  presentation  of  the  synthetic  target  would  eliminate  much 
of  this  problem,  but  such  a  capability  has  not  been  demonstreted. 


Pilots  who  have  flown  OBS  think  It  has  good  training  potential,  but 
not  at  total  sacrifice  of  "real"  (tactical  or  live-fire)  training. 
Also,  the  coat  of  installing  OBS  on  operational  aircraft  must  be 
considered.  An  OBS  purchase  would  reduce  the  amount  of  available 
training  funds.  Cost  savings  must  more  than  account  for  Che  capital 
investment  to  make  a  purchase  worthwhile. 

It  is  obvious  that  a  mors  detailed  analysis  of  relevant  parame¬ 
ters  Is  needed  before  a  decision  can  be  made  about  applying  OBS  to 
operational  air-to-air  gunnery  training.  A  systematic  method  Is 
needed  to  weight  the  advantages  and  disadvantages  of  OBS.  Decision 
theory  will  be  applied  to  the  problem. 

1 . 2  Approach 

Research  began  with  a  thorough  literature  search  of  onboard 
simulation  concepts  and  applications.  References  describing  decision 
theory  methods  were  used  to  determine  the  type  of  analysis  to  be 
applied. 

OBS  is  Introduced  to  the  reader  with  a  detailed  description  of 
this  new  concept.  A  survey  of  the  IF7C  program  was  performed  to 
develop  an  estimate  of  cost  savings  attributed  to  OBS  and  to  identify 
useful  applications.  Data  was  based  on  information  provided  by  the 
contractor  program  manager  and  from  a  detailed  review  of  individual 
flight  test  reports.  Flight  test  cost  and  performance  data  formed  a 
needed  baseline  for  further  analysis. 

The  main  issue  examined  in  this  study  concsrned  a  possible 
decision  by  the  Air  Force  to  Invest  in  OBS  for  retrofit  to  operation¬ 
al  fighter  aircraft  as  a  pilot  training  device.  Many  Issues  were 


taken  Into  account  before  a  specific  course  of  action  vas 
recommended.  To  begin,  objectives  of  the  decision  process  were 
developed  and  pertinent  attributes  selected  for  the  objectives. 
Analysis  considered  purchase  cost,  operating  cost,  effectiveness  of 
training,  pilot  preference,  and  other  appropriate  factors.  A  model 
of  current  Air  Force  training  will  be  developed  from  available 
Information  and  was  serve  as  the  standard  of  comparison  from  which 
alternative  options  were  assessed. 

Alternatives  considered  began  with  keeping  training  as  it 
currently  exists,  progressed  to  increasing  involvement  of  0B8,  and 
concluded  with  an  option  for  converting  all  training  to  0B8,  , 
Expected  utility  was  calculated  for  all  alternatives  and  used  es  the 
final  selection  criteria.  A  sensitivity  analysis  vas  performed  to 
establish  further  confidence  In  the  results.  Final  conclusions 
determine  if  OBS  deserves  consideration  for  major  Air  Force 
application. 


CHAPTER  2 


REVIEW  OF  THE  LITERATURE 

2.1  Onboard  Simulation 

Pllotad  flight  simulators  generally  conalat  of  aoma  form  of 
ground  baaad  davlea  which  attampta  to  realistically  rapraaant  tha 
condition*  of  flight.  Tha  laval  of  sophistication  may  vary  from  a 
simp la  dask-top  unit  to  a  llfe-ais*  cockpit  rapllca  complata  with 
motion  cue*  and  llva-llka  visual  scanas.  Tha  fldallty  or  raallsm  of 
flight  simulation  has  maturad  to  tha  point  that  a  significant  portion 
of  commarclal  alrlina  pilot  training  la  parformad  li.  simulation. 
However.  accurata  rapraaantatlon  of  highly  dynamic  aarlal  combat 
scanarlos  ramalna  a  challanga  avan  to  tha  most  sophlstlcatad  state-of- 
the-art  simulators.  Onboard  simulation  offars  a  naw  approach  In 
combat  simulation  by  flying  tha  pilot  In  a  raal  aircraft  and  simula¬ 
ting  only  tha  axtarnal  conditions  such  as  targat  aircraft.  By  using 
actual  flight  conditions  and  dynamics,  much  of  tha  simulation  burdan 
Is  alimlnatad.  Mora  datallad  discussions  of  OBS  ara  prasantad  latar 
In  this  chaptsr. 

2.1.1  Background 

Tha  concapt  of  Inflight  onboard  simulation  (OBS)  was  first 
damonstratad  during  tha  Unltad  States  Air  Force  Integrated  Flight  and 
Flra  Control  advanced  development  program.  Flight  tasting  of  tha 
IFFC  aystam  occurred  between  July  19B1  and  January  1983.  Primary 
objectives  of  the  program  ware  to  demonstrate  the  feasibility  of 
Integrating  the  flight  and  firs  control  systems  of  a  flghtar 
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aircraft  to  achieve  Improved  weapon  delivery  performance  in  air-to- 
air  gunnery  (AAG) ,  air-to-ground  gunnery  (AGG)  and  bombing  (BMG) . 
Integration  consisted  of  using  fire  control  error  signals  as  inputs 
to  the  flight  control  system  which ,  when  coupled,  would  control  the 
aircraft  to  null  the  errors  and  keep  the  weapon  line  bearing  upon  the 
target.  Landy  (12)  and  Sims  (17)  provide  excellent  overviews  of  the 
IFFC  program. 

OBS  was  originally  conceived  as  an  alternative  to  developing  a 
dedicated  IFFC  hot  bench  unit.  A  hot  bench  le  a  teat  atand  which  can 
be  parked  on  the  ground  next  to  the  aircraft  and  la  used  to  generate 
teat  signals  for  checkout  of  aircraft  susby stems.  A  unique,  and  thus 
expensive,  hot  bench  would  have  bean  designed  and  developed  to  meet 
specific  ground  teat  requirements  of  the  IFFC  ayatam.  OBB  appeared 
to  be  more  attractive  than  a  hot  bench  for  several  reaaona.  OBB  waa 
expected  to  perform  complete  closed  loop  system  test  functions  with 
the  added  advantage  of  being  self-contained  to  the  aircraft,  A  small 
junction  box  was  needed  to  simulate  sensor  inputs  but  otherwise,  OBS 
consisted  entirely  of  software  in  one  of  the  aircraft  digital  compu¬ 
ters.  Cost  of  an  OBS  system  was  expected  to  be  much  lees  than  a 
dedicated  hot  bench.  An  even  greater  advantage  of  OBS  was  that.  It 
would  not  be  restricted  to  ground  use,  but  would  be  operable  in  up 
and  away  flight.  This  advantage  was  significant,  but  the  full 
potential  of  inflight  0B8  was  not  realised  until  flight  testing  was 
begun. 

Once  IFFC  flight  test  was  underway,  experience  with  OBS  grew  and 
software  modifications  wars  made  to  expand  its  capability.  Engineers 


quickly  learned  to  uae  OBS  ae  a  hardware-ln-the-loop  teat  facility 
for  complete  closed  loop  weapon  delivery  teata  during  pre-flight 
checkouts.  IFFC  software  changes  could  be  tested  on  the  aircraft 
prior  to  flight.  Pilots  were  able  to  become  familiar  with  total 
systsm  operation  using  OBS  on  the  ground  and  in  the  air.  OBS  was  a 
valuable  tool  for  IFFC  system  development  because  weapon  delivery 
encounters  could  be  performed  rapidly  and  could  be  duplicated.  Much 
more  data  could  be  collected  per  flight  than  was  possible  using 
actual  targets  with  as  many  as  83  OBS  encounters  recorded  In  one 
flight  (10). 

After  24  IFFC  development  flights  had  bean  completed,  the  electro 
optical  target  sensor  was  damaged.  Without  thlo  sensor,  tactical 
encounters  with  real  targets  could  not  bo  performed,  since  there  was 
no  way  for  the  IFFC  aircraft  to  know  target  position.  OBS  suddenly 
became  the  only  means  of  testing  until  the  sansor  could  be  repaired. 
In  the  interim,  eighteen  development  flights  were  successfully 
completed.  After  tactical  testing  resumed,  an  AAG  flight  was  per¬ 
formed  using  an  F— 1 06  target  aircraft.  During  this  flight  the  IFFC 
pilot  commented,  "This  is  just  like  flying  against  the  OBS  target" 
(14).  Such  a  comment  endorses  the  realism  of  OBS  and  lands  credibil¬ 
ity  to  the  Idee  of  applying  OBS  to  operational  pilot  training. 

2.1.2  Description 

Onboard  Simulation  provides  simulated  targets  for  both  the 
air-to-air  and  air-to-ground  modes.  In  the  air-to-air  gunnery  mode, 
a  delta  wing  aircraft  in  projected  in  the  head-up  display  HUD.  This 
target  changes  sice  and  orientation  in  accordance  with  the  relative 


geometry  between  the  target  and  attacker  aB  perceived  from  the 
attacker  viewpoint.  Target  maneuvers  can  be  varied  from  elmple 
straight  and  level  flight  to  very  dynamic  evasive  maneuvers.  The 
level  of  difficulty  can  be  controlled  for  the  appropriate  skill  of 
the  pilot.  Many  different  encounter*  can  be  stored  in  computer 
memory  and  randomly  presented  (if  desired)  so  the  pilot  cannot 
anticipate  the  type  of  maneuver  to  be  performed.  When  the  attacking 
pilot  is  in  position  to  fire  at  the  target,  onboard  scoring  Is  used 
to  statistically  score  the  expected  number  of  hits  on  target.  The 
bullet  scoring  model  is  contained  In  the  Air  Combat  Evaluator  (ACE) 
software.  A  Bullete-at-Target.-Rangs  (BATR)  symbol,  sometimes  called 
a  hot  point,  is  displayed  to  tha  pilot  so  he  gets  immediate  feedback 
on  where  his  bullet  stream  went  relative  to  the  target.  Hits  will  be 
scored  If  tha  BATR  la  superimposed  on  the  target.  Scoring  la  acti¬ 
vated  whenever  the  trigger  is  depressed,  and  can  also  be  used  in 
simulated  firings  at  real  aircraft  targets.  Figure  2-1  (10)  is  an 
example  of  IFFC  HUD  symbology  Including  the  synthetic  target  and 
scoring  features. 

In  the  air-to-ground  modes,  a  target  symbol  Is  assigned  a  fixed 
three  dimensional  point  in  space  and  Is  displayed  accordingly  In  tha 
HUD.  Absolute  target  position  does  not  change,  but  the  symbol  moves 
In  the  HUD  to  account  for  aircraft  motion.  BATR  and  ACE  scoring  are 
used  for  air-to-ground  gunnery  attacks  to  score  predicted  bullet 
hits,  whereas  estimated  miss  distance  Is  computed  for  bombing  at¬ 
tacks.  Altitude  of  the  eynthsthic  ground  target  can  be  set  at  any 
desired  altitude  so  that  the  weapon  delivery  encounters  can  be 
performed  at  safe  altitudes  without  the  risk  of  flying  into  the 
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ground.  Several  different  encounters  can  be  stored  In  computer 
memory. 

OBS  possesses  many  advantages  for  airborne  pilot  training  and 
testing  of  aircraft  weapon  systems.  It  significantly  reduces  the 
requirement  for  expensive  real  targets  needed  during  checkout, 
validation,  and  evaluation  of  a  system.  Inflight  OBS  is  very  realis¬ 
tic  because  only  the  target  and  target  sensors  are  being  simulated. 
The  attacking  aircraft  uses  all  onboard  subsystems,  such  as  fire 
control,  flight  control,  motion  sensors,  etc.,  in  the  same  manner  as 
in  attacking  real  targets.  The  pilot  actually  flies  the  aircraft  and 
experiences  all  of  the  motion  sensations  of  the  real  flight 
environment.  Many  potential  flight  hazards  are  eliminated  with  the 
synthetic  target.  Head-on  air-to-air  gunnery  encounters  can  be 
performed  with  no  risk  of  mid-air  collision.  Onboard  scoring 
eliminates  the  requirement  to  use  live  weapons  wltn  the  associated 
safety  problems.  Many  more  encounters  can  be  performed  per  flight 
hour  with  OBS,  because  after  each  weapon  encounter  is  completed,  a 
simple  Initial  condition  reset  is  all  that  is  needed  to  prepare  for 
another  encounter.  When  using  real  targets,  lengthy  set-up  time  is 
required  to  properly  position  the  target  and  attacker,  resulting  in 
fewer  encounters  per  flight.  During  the  IFFC  program,  OBS 
demonstrated  all  of  these  advantages.  The  next  chapter  will  develop 
an  estimate  of  cost  savings  attributed  to  OBS. 

2.]. 3  Potential  Future  Applications 

The  IFFC  mechanization  of  OBS  may  have  only  scratched  the 
surface  of  a  much  larger  potential.  It  would  be  possible  to  increase 


the  benefits  attainable  with  OBS  by  extending  the  capability  to 
simulate  total  mission  requirements.  Air-to-Air  OBS  is  currently 
limited  to  short  range  gun  encounters  where  the  target  remains  within 
visual  range.  An  Interface  with  long  range  radar  displays  could  be 
used  to  display  a  synthetic  target  at  maximum  sensor  detection  range. 
Then,  as  the  encounter  progresses,  the  attacker  could  simulate  long 
range  missile  attack,  within  visual  range  missile  attack,  and 
finally,  short  range  gun  attack.  This  total  mission  capability  would 
have  even  greater  benefit  as  a  training  device. 

OBS  target  presentation  could  be  Improved  in  several  ways.  An 
interactive  target  model  has  been  developed  for  ground  based  simula¬ 
tors  which  allows  the  target  to  be  reactive  to  the  attacker  maneu¬ 
vers.  Applying  this  type  of  model  to  OBS  would  allow  the  synthetic 
target  to  behave  very  realistically  like  an  enemy  who  is  trying  to 
avoid  being  killed.  A  helmet  mounted  display  would  enable  the  target 
to  be  displayed  at  any  appropriate  aspect  angle,  eliminating  the  HUD 
fleld-of-view  restriction  of  the  current  OBS.  Several  targets  might 
be  displayed  simultaneously  to  represent  the  multiple  target  aerial 
engagement  which  is  difficult  and  expensive  to  represent  in  training 
with  real  target  aircraft,  but  would  be  expected  in  actual  combat 
conditions.  Heinczman  (6)  refers  to  this  advanced  form  of  OBS  as 
"Embedded  Training  (ET)".  Difficulty  in  accurately  simulating  the 
tactical  fighter  environment  in  ground  based  simulators  makes  ET  an 
attractive  training  method.  It  can  provide  a  very  realistic  environ¬ 
ment  where  it  is  safe  to  fire  simulated  weapons  against  a  synthetic 
target. 


Enhanced  OBS  or  ET  can  be  related  to  the  Air  Combat  Maneuvering 
Instrumentation  (ACMI)  training  aid  which  is  used  by  the  Air  Force 
for  air-to-air  missile  training.  Two  or  more  aircraft  engage  in  mock 
aerial  engagements  while  onboard  Instrumentation  telemeters  aircraft 
state  data  to  ground  receivers.  Ground  systems  display  the  encounter 
In  real  time  to  training  officers  who  monitor  the  encounter.  Viewing 
options  allow  the  encounter  to  be  displayed  from  any  vantage  point. 
All  information  is  recorded  for  pilot  debriefing  after  the  mission. 
"This  is  really  the  biggest  advantage  to  a  student  pilot.  His  aerial 
engagement  is  reconstructed  in  complete  detail... so  he  can  analyze 
his  performance  and  pick  out  mistakes  45  minutes  to  an  hour  later" 
(1:77).  Enhanced  OBS  Is  even  better  than  ACMI  because  it  provides 
the  pilot  with  feedback  during  the  encounter  so  he  can  assess  his 
performance  In  real  time.  In  addition,  he  can  quickly  repeat  the 
very  same  encounter  simply  by  resetting  to  the  initial  condition. 
Repeating  the  encounter  allows  the  pilot  to  apply  experience  gained 
from  the  previous  attempt  and  he  does  not  have  to  wait  until  after 
the  flight.  All  encounters  can  be  recorded  onboard  the  aircraft  for 
post-flight  review  and  comparability  with  ACMI  real-time  ground 
displays  would  be  maintained. 

It  appears  that  OBS  has  significant  growth  potential  to  become  a 
full  mission  training  aid  for  tactical  fighter  pilots.  A  high  level 
of  realistic  training  can  be  provided  safely  at  lower  cost  than 
currently  possible  using  real  aircraft  targets.  The  OBS  capability 
as  demonstrated  in  the  IFFC  program,  is  a  significant  step  toward 
achieving  the  full  potential  of  this  new  technology. 
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2.2  Decision  Analysis  Literature 

"Decision  analysis  has  emerged  from  theory  to  practice  to  form  a 
discipline  for  balancing  the  many  factors  that  bear  upon  a  decision" 
(8:211).  The  basic  purpose  of  using  decision  analysis  is  to  develop 
an  appropriate  course  of  action  from  a  set  of  defined  alternatives 
which  will  provide  the  best  possible  outcome.  Utility  theory  la  a 
specific  form  of  decision  analysis  which  "indicates  that  the  best 
possible  alternative  to  take  is  the  one  yielding  the  highest  possible 
expected  utility"  (16: BO).  The  utility  or  value  of  all  possible 
outcomes  is  established  by  identifying  all  of  the  attributes ,  or 
evaluation  criteria,  which  are  significant  to  the  decision.  Weights 
or  scaling  factors  are  assigned  (see  chapter  6)  for  each  attribute  to 
determine  the  relative  level  of  importance  between  attlbutea.  The 
expected  utility  of  a  particular  alternative  is  the  weighted  average 
of  all  attributes  for  that  specific  alternative. 

Utility  of  any  given  attribute  represents  a  decision  maker's 
preference  for  one  attlbute  value  versus  another.  Various  procedures 
(8,9,15,18)  exist  for  evaluating  utility  functions  because,  "assess¬ 
ment  of  utility  functions  is  as  much  an  art  as  it  is  a  science  and, 
therefore,  no  single  set  of  rules  can  be  laid  down  that  Invariably 
result  in  a  utility  function"  (9:188).  A  typical  utility  function 
ranges  in  value  from  a  preference  limit  of  1.0  at  the  most  desirable 
attribute  level,  and  0,0  at  the  least  desirable  level.  Characteris¬ 
tics  of  the  continuous  function  between  these  endpoints  Indicate  a 
decision  maker's  attitude  toward  risk.  A  linear  relationship  repre¬ 
sents  a  risk  neutral  attitude  where  the  decision  maker  is  indifferent 


between  accepting  the  expected  value  or  the  consequence  of  a  fair 
50-50  lottery.  Indifference  is  defined  such  that  "if  a  decision 
maker  is  Indifferent  between  two  alternatives,  the  expected  utility 
of  the  alternatives  is  the  same"  (18:125).  Risk  proneness  exists  if 
the  decision  maker  is  willing  to  pay  a  premium  to  engage  in  the 
lottery  or,  in  other  words,  if  he  prefers  the  lottery  to  the  expected 
value.  If  a  decision  maker  requires  a  "risk  discount"  (15:271),  he 
is  considered  risk  averee  and  prefers  the  expected  value  to  the 
potential  consequences  of  a  lottery.  Typical  utility  curves  are 
presented  in  figure  2-2. 

A  major  advantage  of  utility  analysis  la  the  ability  to  analyte 
a  situation  where  the  outcome  is  affected  by  multiple  attributes, 
each  of  which  might  be  measured  in  different  units.  Attributes  can 
be  described  in  terms  of  coat,  time,  safety,  effectiveness,  public 
opinion,  and  so  on.  Assignment  of  an  independent  utility  function 
for  each  attribute  provides  a  basis  for  converting  all  the  attributes 
into  a  common  frame  of  reference.  This  provides  the  analyst  with  a 
great  deal  of  flexibility  in  considering  a  broad  range  of  different 
attributes  which  may  affect  a  decision  outcome. 

To  begin  a  decision  analysis,  the  analyst  must  be  able  to 
identify  the  problem  and  establish  meaningful  boundaries.  A  problem 
which  is  too  vague  or  too  broad  will  not  converge  to  a  best  decision 
alternative.  A  set  of  clearly  Btated  objectives  must  bs  developed 
with  descriptors  which  point  the  direction  for  the  snslysls,  such  as 
to  maximize  safety  or  minimize  cost.  Caution  must  be  exercised  to 
capture  all  significant  aspects  of  the  problem  without  using  unneces- 
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sary  or  weak  obj actives  which  quickly  complicate  the  analyaia.  For 
each  objective,  an  attribute  la  eatabliahad  aa  a  meaaure  of  how  wall 
the  objective  is  realized.  Keeney  and  Ralffa  (9:50)  deacriba  desir¬ 
able  properties  of  the  attribute  set  as  follows: 

"It  la  important  in  any  decision  problem  that  the  set  of 
attributes  be  complete,  so  that  it  covers  all  the  important 
aspects  of  the  problem;  operational,  so  that  it  can  be 
meaningfully  used  in  the  analysis;  decomposable,  so  that 
aspects  of  the  evaluation  process  can  be  simplified  by 
breaking  it  down  into  parts;  nonredundant ,  so  that  double 
counting  of  impacts  can  be  avoided;  and  minimal,  so  that 
the  problem  dimension  la  kept  as  small  as  possible." 

The  most  significant  step  in  the  decision  analysis  Is  to  develop 
the  multiple  attribute  utility  function  which  defines  the  weighted 
average  of  the  individual  attributes.  Without  any  simplifying 
assumptions,  this  function  becomes  unmanageably  complex  in  order  to 
account  for  every  possible  interaction  between  the  number  (n)  of 
attributes  Involved.  An  assumption  of  Independence  among  the  attri¬ 
butes  allows  the  collective  assessment  of  n  individual  attribute 
utility  functions  rather  than  a  complex  n-attribute  function.  Two 
conditions  must  hold  to  maintain  attribute  independence:  1)  Given 
any  pair  of  attributes  from  the  total  set  of  n  attributes,  the 
decision  maker's  preference  between  the  pair  must  not  be  affected  by 
the  value  of  the  other  n-2  attributes  and,  2)  the  utility  function 
associated  with  any  individual  attribute  is  not  dependent  upon  values 
of  the  other  n-1  attributes.  Experience  has  shown  that  when  systema¬ 
tically  coping  with  four  or  more  attributes,  the  independence  require¬ 
ment  is  robust  enough  that  "even  when  the  prerequisite  assumptions  do 
not  precisely  hold  over  the  domains  of  the  attributes,  it  may  be  a 
good  approximation  to  assume  they  do"  (9:298).  Independence  was  a 
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factor  In  selecting  the  form  of  multiple  attribute  utility  function 
uaed  in  thla  caae  study. 

Several  examples  of  multiple  attribute  utility  analyaia  applica¬ 
tion  ara  available  in  the  literature.  Thraa  examples  are  cited  here 
to  highlight  the  broad  and  diverse  subject  material  to  which  tha 
analyaia  can  be  applied.  Moakowltz,  Evans  and  Jimenas-Lerma  (16) 
explore  options  for  expansion  of  electrical  generation  capability; 
Keeney  and  Ralffa  (9)  present  a  case  study  for  development  of  airport 
facilities  In  Mexico  City,  Mexico;  and  Brown  (2)  considers  criteria 
to  determine  technology  priorities  for  research  and  development.  In 
each  caae,  a  multiple  eet  of  diverse  objectives  Is  considered  in  a 
decision  process  of  selecting  a  course  of  action  with  the  hlgheet 
expected  utility. 

Keeney  and  Ralffa  (9)  have  compiled  a  very  comprehensive  treat¬ 
ise  of  the  multiple  objective  decision  problem  with  a  generalized 
procedure  for  performing  a  structured  decision  analyaia,  Thla  paper 
applies  the  procedure  to  demonstrate  a  multiple  utility  analysis  for 
application  of  onboard  simulation  to  operational  flight  training. 
Further  explanation  of  the  procedure  is  provided  In  conjunction  with 
the  analysle. 


20 


CHAPTER  3 


THE  VALUE  OF  OBS:  IFFC  PROGRAM  EXPERIENCE 


3.1  Software  Development  Savings 

The  moat  obvious  method  to  establish  an  estimate  of  the  value  of 
OBS  Is  to  examine  results  of  testing  during  the  IFFC  program.  Cost 
savings  which  can  bs  attributed  to  the  presence  of  OBS  can  be  identi¬ 
fied  and  a  total  savings  computed.  An  interesting  fact  vaa  discovered 
which  strongly  indicates  that  OBS  can  directly  reduce  the  cost  of 
software  development.  According  to  Westarmair  (22),  the  average  or 
typical  cost  of  software  development  for  advanced  development 
programs  like  IFFC  is  eight  to  ten  manhours  per  word  of  software 
code.  A  final  contract  cost  tabulation  of  the  IFFC  program  revealed 
an  average  of  only  three  and  one-half  manhours  charged  per  word  of 
software.  This  very  significant  difference  is  attributed  to  the 
presence  of  OBS  for  reasons  discussed  in  the  next  paragraph. 

Software  development  for  the  IFFC  program  involved  use  of  a 
software  test  facility  and  man-ln-tha-loop  simulation  at  the  company 
facilities  in  St  Louis,  Missouri.  Here,  software  was  developed  into 
an  operational  flight  program  for  the  flight  test  phase  of  the 
program.  Initial  airworthiness  testing  was  slso  performed  at  St 
Louis,  after  which  the  aircraft  was  transferred  to  Edwarda  AFB, 
California,  for  75  development  flights.  Throughout  flight  tasting, 
the  software  test  facility  and  simulator  wers  used  to  support  efforts 
to  debug  the  flight  software.  The  OBS  model  was  used  at  ail  faclli- 
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ties  at  «  common  baseline  against  which  software  performance  could  be 
tasted.  This  common  OBS  link  greatly  increased  the  efficiency  of 
effort  between  engineers  at  the  threo  different  facilities  since 
anomalies  could  usually  be  repeated  and  quickly  idantifiad.  Without 
OBS,  a  problem  which  is  identified  in  flight  test  is  often  difficult 
to  reproduce  in  support  facilities.  OBS  also  permitted  engineers  to 
themselves  fly  the  weapon  delivery  encounters  on  the  ground  onboard 
the  aircraft  and  quickly  assess  the  Impact  of  changes.  This  reduced 
dependence  upon  flight  recordings  or  pilot  comments  to  identify 
problems.  Many  problems  were  identified  during  OBS  ground  teste 
which  otherwise  would  have  been  found  in  flight.  Test  flights  were 
generally  more  successful  than  would  be  possible  without  OBS. 
Approximately  350  total  software  changes  were  performed  during  the  75 
development  flights, 

A  total  of  33,000  words  of  ooftware  were  developed  for  the  IFFC 
flight  program.  Of  this  amount,  5,300  words  were  involved  in  OBS 
plus  BATR  and  ACE  scoring,  leaving  27,700  words  of  IFFC  fire  and 
flight  control.  Using  a  cost  of  $60  per  manhour,  software  cost 
savings  due  to  the  presence  of  OBS  can  be  computed  by  comparing 
software  cost  at  typical  rates  and  at  the  reduced  rate. 

Typical  software  cost: 

(27,700  words)(8  manhours /word) ($60 /manhour)"  $13,296,000 
NOTE:  The  value  of  eight  manhours  per  word  is  the  most  conser¬ 
vative  value  in  the  range  of  sight  to  ten. 


Cost  with  OBS: 

(27.700  words)(3.5  manhour* /word) ($60/manhour)«  $5,817,000 
Coat  Dif farance:  $13,296,000  -  $5,817,000  -  $7,479,000 

Coata  incurred  In  tha  development  of  OBS  ahall  now  be  aubtracted 
from  the  eavlnga  In  IFFC  aoftwara  development.  In  addition  to  tha 
5,300  worde  of  aoftwara,  an  onboard  almulatlon  junction  box  was 
required  for  ground  uae  of  0B8,  and  a  modification  to  tha  HUD  procea- 
aor  waa  required  for  dloplay  of  tha  new  aymbology. 

OBS  development  coat: 

(5,300  worda)(3.5  manhour a /word) ($60/manhour)" 

Junction  box  ■ 

Modification  to  HUD  proceaaor  (3  aata)  » 

Total  * 

The  total  net  aoftware  saving*  attributed  to  OBS  ia  computed  aa  the 
difference  ba tween  dollar*  saved  and  cost  to  develop  OBS. 

$7,479,000  -  $1,213,000  -  $6,266,000 

3.2  Teat  and  Evaluation  Savings 

OBS  directly  contributed  to  reduced  cost  of  flight  test  and 
evaluation  of  the  IFFC  eyatera  in  two  way*.  Firet,  synthetic  targets 
saved  the  expense  of  real  targets  and,  second,  more  test  events  per 
flight  could  be  performed  using  OBS.  Definition  of  test  modes  is 


$1,113,000 

10,000 

_ 90.000 

$1,213,000 
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warranted  before  attempting  to  estimate  savings.  Live-fire,  tacti¬ 
cal,  and  OBS  testing  was  performed  for  each  of  the  three  weapon 
delivery  modes  (AAG,  AGG,  and  BMG)  giving  a  total  of  nine  different 
conditions.  Each  condition  la  described  as  follows: 

1.  Live-fire  AAG:  An  aerial  tow  target  was  towed  behind  an  F-4 
aircraft  while  the  attacking  F-15  aircraft  fired  live  20 
millimeter  ammunition,  attempting  to  hit  the  tow  target. 

2.  Tactical  AAG:  Aircraft  such  as  T-38,  F- 106,  and  A- 7  ware 
flown  aa  dynamic  maneuvering  targets  for  the  attacking  F-13. 
No  bullets  wars  actually  fired,  but  onboard  scoring  was  used 
to  statistically  model  the  expected  number  of  hits. 

3.  OBS  AAG:  Synthetic  target  presented  in  HUD  for  F-13  attack. 
Expected  hits  were  determined  by  onboard  scoring. 

4.  Live-fire  AGO:  20  millimeter  ammunition  was  fired  by  the 
F-13  at  stationary  cloth  banners  which  wars  anchored  to  the 
ground.  Hits  were  scored  by  counting  bullet  holes  in  the 
banner. 

3.  Tactical  AGG  :  Non-firing  gun  attack  at  cloth  banners  which 
were  anchored  to  the  ground.  Expected  hits  were  determined 
by  onboard  scoring. 

6.  OBS  AGG  :  Non-firing  gun  attack  against  a  stationary 
synthetic  ground  target.  Expected  hits  were  determined  by 
onboard  scoring. 

7.  Live-fire  BMG  :  BDU-33  practice  bombs  were  dropped  by  the 
F-15  on  standard  bombing  range  target.  Miss  distance  was 


measured  by  a  survey  of  bomb  Impact  relative  to  the  range 
bullseye. 

8.  Tactical  BMG  :  Bombing  attack  without  bomb  release  against 
a  standard  bombing  range  target.  Mies  distance  was  estimat¬ 
ed  by  onboard  scoring. 

9.  0B8  BMG  t  Bombing  attack  without  bomb  release  against  a 
synthetic  ground  target.  Miss  distance  was  estimated  by 
onboard  scoring. 

A  detailed  summary  of  all  encounters  flown  in  the  1FFC  teat 
program  Is  presented  in  the  Appendix.  Data  was  obtained  from  indivi¬ 
dual  flight  reports  from  all  75  development  flights  (13).  These  data 
will  be  needed  to  estimate  the  test  and  evaluation  savings  directly 
attributed  to  OBS.  Cost  will  be  based  on  the  1983  fiscal  year 
reimbursement  rates  which  the  Air  Force  Flight  Test  Center  charged  to 
the  IFFC  program.  That  is  approximately  $8,500  per  F— 15  or  F-4 
flight  hour  and  $1,000  per  T-38  flight  hour.  T-38  aircraft  were  used 
as  target  aircraft  aa  much  as  possible  because  they  provided  fighter 
class  performance  at  the  lowest  possible  cost. 

Additional  Information  which  will  be  needed  is  the  average  number 
of  gun  encounters  performed  per  flight  hour  of  live-fire,  tactical, 
or  OBS  testing.  Flights  which  ware  dedicated  to  each  particular  mode 
of  Interest  were  used  to  compute  the  avergs  values  and  are  presented 
in  Table  3-1.  Flights  which  occurred  very  early  in  the  program  were 
not  used  in  the  estimates  because  test  procedures  were  being  perfec¬ 
ted  and  pilot  learning  effects  would  bias  the  data. 
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From  the  Appendix ,  the  total  number  o£  OBS  teat  encounters  can 
also  be  tabulated:  647  AAG,  191  AGG,  and  137  BMG.  Now  OBS  cost 
savings  can  be  computed  for  each  weapon  mode. 

3.2.1  Air-to-Air  Gunnery 

At  an  average  of  40  OBS  events  per  flight  hour*  16.2  flight 
hours  in  the  F-15  were  needed  to  log  647  events. 

(647  events)/(40  events/flight  hour)*  16.2  flight  hours 

Total  cost  for  16.2  hours  of  F-15  time  is  $137,700. 

(16.2  hours) ($8, 500/hour)  *  $137,000 

If  this  many  events  had  been  performed  in  the  tactical  mode  using  a 
T-38  aircraft  at  an  average  of  16  events  per  flight  hour,  40.4  flight 
hours  would  have  been  needed. 

(647  events)/(16  events/flight  hour)*  40.4  flight  hours 

The  combined  cost  of  an  F-15  and  T-38  per  flight  hour  is  $9,500,  for 
a  total  of  $383,800  for  40.4  flight  hours.  Net  savings  for  using  OBS 
is  then  the  difference  of  $246,100,  Remember  that  Mie  Inexpensive 
T-38  was  used  for  a  tactical  target.  If  an  F-15  had  been  used  as  a 
target,  an  even  greater  cost  difference  would  have  been  experienced 
($549,100). 
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3 . 2  4  2  Air-to-Ground  Gunnery 

For  AGG,  savings  were  accrued  because  more  encounters  vers  flown 
per  hour  than  in  the  tactical  mode.  The  191  AGG  encounters  required 
4.8  flight  hours  using  OBS  at  a  cost  of  $40,800. 

(191  events)/(40  events/flight  hour)*  4.8  flight  hours 
(4.8  flight  hours) ($8, 500/flight  hour)-  $40,800 

In  a  tactical  mode.  11.9  flight  hours  would  have  been  required  at  a 
cost  of  $101,150. 

(191  event8)/(16  events/flight  hour)  -  11.9  flight  hours 
(11.9  flight  hours) ($8, 500/flight  hour)  -  $101,150 
OBS  savings:  $101,150  -  $40,800  -  $60,350 

It  has  been  assumed  that  onboard  scoring  can  be  used  in  the  tactical 
AAG  and  AGG  modes  to  determine  hits  on  target.  Without  onboard, 
scoring,  live  bullets  would  have  been  fired  at  an  additional  cost  of 
$3,760  per  flight  (940  rounds  @  $4.00  per  round). 

3.2.3  Bombing 

In  the  BMG  mode,  137  IFFC  OBS  encounters  were  performed.  Again, 
savings  are  achieved  using  OBS  because  more  encounters  can  be  per¬ 
formed  per  flight  hour.  These  137  encounters  required  3.4  flight 
hours  at  a  cost  of  $28,900. 
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(137  encounters)/ (40  encounters /flight  hour)*  3.4  flight  hours 
(3.4  flight  hours) ($8, 500/f light  hour)  ■  $28,900 

Tactical  bombing  encounters  would  have  required  8.6  flight-hours  at 
$73,100. 

(137  encounters) /( 16  encounters/fllght  hour)*  8.6  flight  hours 
(8.6  flight  hours) ($8, 500/flight  hour)  ■  $73,100 

The  OBS  savings  difference  Is  $44,200. 

$73,100  -  $28,900  -  $44,200 

Again  note  that  without  onboard  scoring,  137  practice  bomba  at  $20 
each  would  have  coat  an  additional  $2,740.  Total  test  and  evaluation 
savings  due  to  OBS  was  $350,650. 

$246,100  +  $60,350  +  $44,200  -  $350,650 

3.3  Total  Savings 

When  software  development  and  ceet  and  evaluation  cost  savings 
are  combined,  the  total  savings  to  the  IFFC  program  was  $6,616,650. 
Compared  to  the  cast  to  develop  OBS  ($1,213,000),  a  savings  of  5.5 
times  the  development  cost  was  realised.  This  is  a  vary  significant 
rate  of  return.  Total  coot  for  the  IFFC  program  was  approximately 
$25,000,000.  Based  on  the  computed  savings,  the  program  would  have 
cost  $31,616,650  or  26.5  per  cent  higher  without  OBS. 
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3.4  Additional  Factors 

In  addition  to  the  savings  calculated  so  far,  OBS  contributed  to 
the  IFFC  program  In  other  more  subtle  ways  which  are  difficult  to 
measure  In  terms  of  dollars.  Using  the  ground  mode  of  OBS  allowed 
pilots  to  "fly"  encounters  and  familiarize  themselves  with  symbology, 
swltchology,  encounter  geometries,  event  sequences,  expected  perfor¬ 
mance,  etc.,  before  actual  flight.  System  engineers  could  observe 
this  ground  training  and  work  with  the  pilot  to  confirm  everything 
was  ready  for  flight.  Without  OBS,  additional  man-in-the-loop 
simulation  and/or  flight  tsst  would  have  been  required. 

For  air-to-ground  attack  (AGG  or  BMG)  risk  of  ground  clobber  is  a 
major  concern,  especially  in  test  programs.  IFFC  automatic  control 
coupling  between  the  flight  and  fire  control  systems  was  a  new 
concept  which  created  additional  unknowns  early  in  the  flight  test. 
Since  OBS  synthetic  ground  targets  can  be  placed  at  any  desired 
altitude,  initial  IFFC  ground  attack  profiles  were  flown  at  safe 
altitudes  until  confidence  in  the  IFFC  system  was  established.  Then 
actual  ground  targets  were  used.  Value  of  this  added  safety  is 
significant,  but  difficult  to  assign  a  dollar  amount. 

Some  air-to-air  gunnery  conditions  are  difficult  or  unsafe  to  fly 
when  using  real  target  aircraft.  Conditions  approaching  head-on  or 
front  quarter  attack  pose  the  potential  of  collision  between  the  two 
aircraft.  For  this  reason,  most  testing  or  operationa:1.  flight 
training  is  restricted  to  aspect  angles  of  135  degrees  or  less  (20). 
Onboard  simulation  can  be  safely  used  to  test  front-quarter  attack 
because  there  is  no  risk  of  collision  with  the  synthetic  target.  It 
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Is  Important  to  be  able  to  teat  systems  or  train  in  the  front  quarter 
because  of  studies  which  Indicate  a  majority  of  combat  encounters 
(where  safety  is  less  significant)  would  actually  occur  (19).  OBS  is 
very  valuable  when  compared  to  the  potential  consequences  of  a 
mid-air  collision. 


31 


CHAPTER  4 


DEFINING  THE  DECISION  PROBLEM 

The  previous  chapter  highlighted  the  value  of  OBS  as  a  tool  to 
support  the  flight  test  development  of  the  IFFC  weapon  system.  A 
natural  question  to  ask  at  this  point  Is,  can  OBS  be  successfully 
applied  to  the  larger  scale  application  of  Air  Force  pilot  training? 
Specifically,  would  OBS  contribute  to  more  effective  and  efficient 
pilot  training  In  the  air-to-air  gunnery  mission  compared  to  current 
training  methods? 

4,1  Decision  Alternatives 

Seversl  alternatives  exist  for  how  to  apply  OBS.  The  baseline 
for  comparing  alternatives  will  be  current  training  methods  without 
any  changes.  Onboard  scoring  can  be  implemented  without  OBS  and  vice 
versa.  However,  OBS  without  scoring  is  not  considered  a  reasonable 
alternative  because  scoring  is  currently  a  problem  which  would 
continue  with  synthetic  targets  if  scoring  Is  not  Included.  Current 
training  augmented  with  only  onboard  scoring  would  be  a  reasonable 
alternative  and  shall  be  considered.  Once  OBS  is  Installed  in  an 
aircraft,  the  relative  amount  of  OBS  training  compared  to  live-fire 
end  tactical  training  would  be  variable  but  the  capital  Investment 
remains  the  same.  Thus,  a  range  of  low  to  high  percentage  of  OBS 
training  shall  be  examined.  Eight  different  alternatives  shall  be 
used  In  the  analysis.  They  are  defined  as  follows: 
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Al;  Continue  current  operational  training  without  any  OBS 
features . 

A2:  Add  only  onboard  scoring  (BATR/ACI)  to  current  training 
procedures. 

A3:  Include  I  OBS  encounter  for  every  2  real  encounters. 

A4:  Include  1  OBS  encounter  for  evary  1  real  encounter. 

A5:  Include  2  OBS  encounters  for  every  1  real  encounter. 

A6:  Include  4  OBS  encounters  for  every  1  real  encounter. 

A7:  Include  6  OBS  encounters  for  every  1  real  encounter. 

AS:  Perform  all  gun  training  using  OBS  encounters. 

The  first  Important  step  is  to  develop  a  modal  of  the  currant 
procedures  for  Air  Force  training  in  the  air-to-air  gun  mode.  Due  to 
the  difficulty  of  detailed  record  keeping)  the  Air  Force  does  not 
record  individual  events  or  encounters  which  take  place  on  training 
missions.  Some  assumptions  and  estimates  are  required  to  construct  a 
modal.  Personnel  from  the  Tactical  Liaison  Office  at  Wrlght-Patter- 
son  Air  Force  Base  were  Instrumental  In  providing  some  necessary  data 
for  constructing  the  model  and  checking  for  reasonableness. 

Most  air-to-air  gun  training  encounters  are  not  performed  on 
dedicated  gun  missions.  Most  often  a  mission  will  involve  several 
training  objectives  such  as  missile  encounters,  formation  flying. 
Intercepts,  and  so  on.  Only  live-fire  missions  against  Dart  targets 
are  dedicated  strictly  to  gun  training.  The  approach  is  to  start 
with  a  log  of  all  missions  flown  in  a  one-ysar  period  and.  based  on 
the  type  of  mission  and  typical  mission  breakdowns,  estimate  the 
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total  numbar  of  gun  events  per  year.  Reference  (21)  lists  all 
missions  flown  by  596  F-15  aircraft  over  a  one-year  period  (June  1981 
through  June  1982).  Only  F-15  aircraft  were  considered  In  the  model. 
Table  4-1  Is  a  summary  of  missions  flown  and  estimated  gun  encounters. 

Important  results  from  Table  4-1  were  reviewed  for  a  reasonable¬ 
ness  check  of  the  molnl.  Each  pilot  receives  approximately  245  gun 
training  events  per  yuar,  25  llvc-flre  and  220  tactical  encounters, 
or  roughly  90  parent  tactical  and  10  percent  live-fire.  This  same 
ratio  of  tactical  and  live-fire  sncounters  was  used  to  define  relative 
percentages  In  all  alternatives.  For  example,  alternative  A-3  calls 
for  one  OBS  encounter  for  every  two  real  encounters.  Ninety  percent 
of  the  real  encounters  are  assumed  tactical,  while  ten  percent  are 
assumed  live-fire. 

4.2  Objectives  and  Measures  of  Effectiveness 

Before  evaluating  alternatives,  effectiveness  measures  need  to 
be  developed  to  form  a  basis  of  comparison.  The  overall  objective  of 
the  comparison  Is  to  determine  which  of  the  eight  possible  alterna¬ 
tives  provides  the  Air  Force  with  the  highest  level  of  pilot  training 
within  reasonable  coat  and  which  is  psychologically  acceptable  to 
pilots.  A  hierarchy  of  objectives  is  shown  in  Figure  4-1.  Based  on 
the  hierarchy,  six  objectives  were  selected: 

1 ,  Minimize  the  average  cost  of  training  measured  in  dollars 
per  gun  event. 

2.  Provide  the  highest  possible  level  of  live-fire  training 
which  familiarizes  pilots  with  gun  system  operation,  confirms  gun 
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TOTAL  F-15  MISSIONS  FLOUW  FROM  JUH  81  -  JUH  82  =  119,712 


IMPROVED 


HIERARCHY  OF  OBJECTIVES 


operability  and  gun  boraslght  alignment.  Number  of  llve-flra  mis¬ 
sions  par  pilot  per  year  la  the  measurement  unit. 

3.  Maximize  the  effectiveness  of  training  for  weapon  effective¬ 
ness;  that  Is,  training  pilots  where  to  aim  and  when  to  fire  to  score 
hits  on  target,  Measured  In  terms  of  fractional  effectiveness  in 
representing  actual  combat  conditions. 

4.  Maximize  effectiveness  of  training  for  aircraft  maneuvering 
tactics;  train  pilot  In  tactics  which  will  allow  him  to  maneuver  into 
position  for  gunshot  opportunity  without  self  exposure.  Measured  in 
terms  of  fractional  effectiveness  in  representing  actual  combat 
conditions. 

3.  Provide  training  psychologically  acceptable  to  pilots;  a 
minimum  level  of  real  encounters  is  desired  by  pilots  after  which  0B8 
encounters  can  be  flown.  Measured  as  number  of  real  encounters  per 
pilot  per  year. 

6.  Maximize  the  availability  of  training  based  on  the  fraction 
of  time  when  planned  events  can  actually  be  performed. 

Safety  of  training  was  considered  as  a  separata  objective,  but 
safety  restrictions  affect  the  ef fectivenesa  of  training  in 
objectives  3  and  4.  Thus,  safety  is  Indirectly  considered. 


CHAPTER  5 


UTILITY  FACTORS 


5.1  Utility  Function  for  Each  Attrlbuta 

To  pvocaad  with  the  analyaia,  It  la  now  nacaaaary  to  aatabllah 
prafarancaa  for  tha  ranga  of  poaalbla  valuta  for  each  attrlbuta. 
Frafarancaa  art  maaaured  In  unlta  of  utility .  where  utility  la 
aaelgnad  valuea  from  aero  to  one.  Maximum  and  minimum  valuta  of  tha 
attrlbuta  ara  to  be  aalacted  which  rapreaant  utilities  of  1  and  0. 
reapectlvely .  Then  ettltudea  of  the  daclalon  maker  toward  rlak 
aatabllah  the  utility  value  over  the  attribute  range.  and  a  utility 
curve  of  all  poaalble  outcomee  la  defined.  In  thla  daclalon 
analyaia.  the  daclalon  maker  la  not  any  alngla  Individual,  but  rather 
the  United  Statea  Air  Force.  For  thla  reaaon.  attitude  toward  rlak 
la  expected  to  be  nearly  neutral  becauae  Individual  tendenclea  toward 
rlak  avaraion  or  rlak  pronenaaa  are  averaged  out.  From  the 
literature  (9) ,  rlak  neutral  utility  functions  ara  typically  linear 
and  monotonlcally  lncraaelng  toward  the  moat  desirable  value  of  the 
attribute.  One  poaalbla  exception  la  the  coat  attribute  where  moat 
decision  makera  tend  to  be  rlak  averse  (15.  18).  A  conatantly  rlak 
averse  exponential  function  la  more  appropriate  here. 

5.1.1  Cost  Attribute  (X^) 

An  extensive  background  muat  be  established  before  tha  coat 
utility  function  can  be  determined.  Thla  includes  the  statement  of 
several  assumptions  uaed  to  derive  coat  figures. 
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1)  a  constant  annual  training  budgat  la  aatabllahad  for  training 
Air  Forca  pllota.  Any  lnvaatmanta  In  training  devices,  Ilka  OBS, 

ara  assumed  to  be  takan  from  tha  axle ting  budgat. 

2)  Inflation  affacta  ara  allmlnatad  by  assuming  constant  dollar 
value. 

3)  Capital  lnvastmant  In  OBS  Is  dlstrlbutad  into  aqual  annual 
paymants  ovar  a  payback  parlod  of  flva  yaars.  Tha  axpactad  sarvlca 
Ufa  of  OBS  would  ba  much  longer,  but  a  eonaervative  approach  la 
takan  by  keeping  tha  payback  to  five  years. 

A)  Payback  la  computed  at  an  annual  interest  rata  of  10  percent. 

5)  Cost  of  operation  for  P—15  or  F-4  aircraft  Is  $8,500  par 
flight  hour,  20mm  ammunition  la  $4.00  par  round,  and  a  Dart  tow 
target  la  $200  par  miaslon. 

6)  Estimated  purchase  cost  of  OBS  la  $12,000  par  F-15  aircraft, 
if  outfitting  the  entire  fleet.  Tha  breakdown  conslsta  of  $5,000  to 
modify  a  circuit  card  In  tha  HUD  and  $7,000  for  5,300  words  of 
software  added  to  tha  aircraft  central  computer.  All  required 
documentation,  such  as  changes  to  aircraft  technical  manuals  and 
operating  procaduras,  ara  Included, 

Tha  baseline  annual  gun  training  budgat  must  be  defined  before 
tha  relative  cost  of  tha  eight  decision  analysis  alternatives  can  ba 
established.  Since  an  official  figure  could  not  ba  found,  a  model 
was  developed  to  estimate  the  annual  cost  of  gun  training.  The  coat 
of  llve-fira  gunnery  can  ba  easily  estimated  because  individual 
missions  are  dedicated  to  gun  training  end  nothing  else.  Using 
information  from  Table  4-1: 
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Average  flight  time:  1.15  hour 

Aircraft:  One  F-15  and  one  F-4  tow  @  $8,500  per  hour  (each) 

CoBt:  (2)  ($8, 500/hour) (1.15  hour)  -  $19,500 
Ammunition:  (940  round*) ($4. 00/round) ■  $3,760 
Dart  target:  ($200/mlaalon) 

Total  mlaalon  coat:  $19,500  +  4,760  +  200  »  $23,460 

Coat  per  event:  ($23,460/miaelon)/(6  event a /mlaalon) -($3, 9 10/event) 

Annual  coat:  (2,993  mlaalon/year) ($23, 460/mlaalon)"$70, 215, 780 

Annual  coat  of  tactical  gun  training  la  much  harder  to  compute 
becauae  miaalona  are  not  dedicated  to  gun  training.  In  Table  4-1,  an 
aasumptlon  waa  made  that  F-15  pilot*  typically  experience  1.8  gun 
encountera  agalnat  each  other  during  mlaalon  typaa  2,  3,  and  4. 

Theae  gun  eventa  are  logged  on  the  way  to  and  from  mlaalla  range*  or 
by  maneuvering  for  a  gun  ahot  after  a  practice  mlaalla  encounter  haa 
transitioned  Into  a  gun  opportunity.  Another  way  of  atatlng  thla  la 
that  for  every  two  F-15  flight*  ,  each  pilot  record*  an  average  of 
1.8  gun  encountera  agalnat  the  other  F-15.  Baaed  on  thla  aa sumption, 
a  total  of  157,301  tactical  gun  encounters  ware  logged  by  715  F-15 
pilot*  over  a  one-year  period.  An  equivalent  number  of  dedicated 
tactical  gunnery  miaalona  can  be  derived  by  dividing  the  total  number 
of  encountera  by  the  average  number  of  tactical  encountera  expected 
par  mlaalon  from  IFFC  program  experience,  that  la,  16  tactical  gun 
events  per  flight  hour.  Since  operational  flights  last  an  average  of 
1.25  flight  hours,  20  tactical  gun  encountera  are  expacted  per 


(157,301  encounters)/ (20  encounters/flight)-7,865.05  flights 

Now  the  cost  of  7,865.05  dedicated  tactical  gunnery  flights  can 
be  computed.  Note  that  an  F-15  is  used  for  target  cost  and  not  a  T-38 

Average  flight  time:  1.25  hours 

Aircraft:  Two  F-15  <?  $8, 500 /hour  each  ■  ($17, 000/hour) 

Total  mission  cost:  (1.25  hour) ($17, 000/hour)  ■  $21,250 
Cost  per  encounter: 

($21 ,250/mlsslon) / (20  encounters /mission)*  ($1 ,062. 50/encounter) 
Yearly  coat:  (7,865.05  miaaiona) ($21,250/misaion)  -  $167,132,310 

Total  yearly  cost  for  live-fire  and  tactical  F-15  gun  training  is 
$237,348,090. 

$167,132,310  +  70,215,780  -  $237,348,090 

Although  this  total  dollar  amount  is  attributed  to  gun  training,  it 
also  contributes  to  basic  pilot  flying  skills  and  aircraft  familiari¬ 
zation  which  eliminate  the  need  for  additional  training  flights.  The 
$237,350,000  Includes  this  prorated  share  of  flying  skills  benefits 
which  contributes  to  total  pilot  experience. 

With  the  constant  annual  dollar  amount  established  for  gun 
tralnlug,  the  relative  cost  of  the  eight  decision  alternatives  can  bs 
computed,  Average  cost  per  encounter  will  be  the  unit  of  comparison. 


5. 1.1.1  Alternative  Al:  For  current  training ,  the  cost  per  encounter 
is  determined  by  dividing  the  cost  of  training  by  the  total  number  of 


encounters: 

17,958  live-fire  encountere  +  157,301  tactical  encounters  * 

175,259  total  encounters 

($237, 3A8,090)/(175, 259  encounters)-  ($l,354.27/encountcr) 

5. 1.1. 2  Alternative  A2:  The  Investment  cast  for  purchase  of  onboard 
ecoring  must  be  considered.  An  annual  equivalent  cost  must  be  subtrac¬ 
ted  from  the  total  available  training  budget.  The  cost  of  only  onboard 
scoring  is  estimated  at  $2,000  per  aircraft.  A  simple  ratio  comparing 
$7,000  for  5,300  words  of  OSS  software  with  1,300  words  of  scoring 
software  gives  a  cost  of  $1,716.  Assuming  some  slight  overhead  Juetl- 
fieu  a  round-off  to  $2,000. 

5,300  words /$7, 000  -  (1,300  words)/X 
Where:  X  -  $1 ,716  -  $2,000 

At  $2,000  per  aircraft,  total  cost  for  596  F-15s  la: 

($2,000X596)  -  $1,192,000 


Annual  equivalent  payment  at  1X0  percent  interest  over  n-5  years  is: 

A-P  (A/P,  i,  n)  »  (1, 192, 000)(0. 1638)  -  $314,450 


Where  (A/P,l,n)  is  an  equal-payment-sarles  capital  rccovary  factor  (5). 
Subtracting  from  the  total  annual  training  budget  gives: 


$237,348,090  -  $314,450  -  $237,033,640 

which  Is  available  for  training  after  the  payment  for  scoring. 

The  number  of  encounters  which  can  be  flown  is: 

($237 , 033,640) / <$ 1 , 354 . 27/encounter )-  175,027  encounters 
Live  encounters  equal  10.25  percent  of  total  (17,940) 

Tactical  encounters  equal  89.75  percent  of  total  (157,087) 

Average  cost  per  encounter  is  found  by  dividing  the  total  training 
budget  by  the  total  number  of  encounters. 

($237,34B,090)/(175,029  encounters)"  ($1,356. 07/encounter) 

5. 1.1. 3  Alternative  A3:  OBS  with  scoring  is  Installed  in  each  air¬ 
craft  and  cne  OBS  gun  encounter  is  flown  for  every  two  real  (llve-flra 
and  tactical)  encounters.  Total  cost  of  OBS  is  $12,000  for  each  of  596 
aircraft. 


($12,000) (596)  »  $7,152,000 

Annual  equivalent  payback  at  1-10  percent  interest  over  n-5  years  is: 

A-P  (A/P,  1,  n)  -  (7,512,000) (0.2638)-  $1,164,346 
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Subtracting  from  the  annual  training  budget  gives: 


$237,348,090  -  $1,164,346  -  $236,183,744 

which  is  available  for  training  after  the  payment  for  OBS. 

Number  and  type  of  encounters  are  all  computed  using  the  cost  of 
$212.50  per  OBS  encounter  as  computed  previously. 


[OBS  cost] 

[tactical  cost] 

[live-fire  coat] 

[total  cost] 

Given  that  x  »  number  of  OBS  events 
Solving  for  x  given:  x  ■  80,850  encounters 

Simple  substitution  also  Identifies: 

145,127  tactical  encounters  +  16,574  live-fire  encounters  ■ 
242,551  total  encounters 

Average  cost  per  encounter  is  found  by  dividing  the  tot.il  training 
budget  by  total  encounters  and  gives: 


[ ($212. 5/event)x] 

[<2x> (0.8975) ($1,C62. 5/event)] 
[  (2x) (0. 125) ($3, 9 10/event ) ] 
$236,183,744 


($237, 348,090)7(242, 551  encounters)"  $978. 55/encounter 


5. 1.1.4  Alternatives  A4  through  A8:  The  sam«  analysis  method  was 
used  to  compute  options  A4  through  A8  and  results  for  all  alterna¬ 
tives  are  presented  In  Table  5-1.  The  annual  equivalent  cost  for 
purchase  of  OBS  is  the  same  for  all  these  alternatives.  Only  the 
ratio  of  OBS  versus  real  encounters  changes. 

As  can  be  seen  from  Table  5-1.  cost  ranges  from  a  low  of  $213.55 
per  encounter  for  alternative  A8  to  a  high  of  $1356.07  per  encounter 
for  alternative  A2.  From  this,  one  could  assume  that  a  cost  of  $100 
per  encounter  might  have  the  highest  utility  (U(100)  -  1)  and  if 
average  cost  grew  to  $2,000  per  encounter,  utility  would  be  minimal 
(U(2.000)  -  0).  Using  these  boundary  values,  the  endpoints  of  the 
cost  utility  function  are  defined.  To  determine  the  level  of  risk 
aversion,  a  certainty  equivalent  is  determined  based  on  a  50-50 
lottery  (100,  2,000)  with  an  axpectad  value  of  1,050.  A  risk  averse 
decision  maker  would  accept  for  certain  a  value  greater  than  the 
expected  value,  rather  than  risk  the  lottery.  A  certainty  equivalent 
of  1,200  was  selected  as  a  representative  value  for  slight  risk 
aversion.  By  simple  curve  fit  the  exponential  function,  U(X^)  ■ 

2.036  -  #0‘000355(Xj)  three  data  points  and  the  utility 

function  has  been  determined.  Figure  5-1  is  a  graph  of  the  function. 
5.1.2  Live-Fire  Gunnery  Attribute  (Xg) 

It  was  previously  determined  that  7-15  pilots  currently  average 
approximately  four  live-fire  training  missions  per  year  or  24  encount¬ 
ers.  For  experienced  pilots,  this  is  enough  to  maintain  live  gunnery 
proficiency.  Inexperienced  pilots  might  be  able  to  use  a  few  more 
encounters  to  improve  their  skills.  Therefore,  six  missions  per  year 
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X,  UTILITY  FUNCTION 


is  selected  aa  th#  most  dsslrabla  avaraga  with  a  utility  of  1.0.  No 
ancountars  par  year  obviously  would  not  contribute  to  pilot  skills  or 
help  confirm  gun  system  operation  and  is  assigned  a  utility  of  zero. 
Assuming  a  linear,  risk  neutral  function  fully  determines  the  utility 
function  as  U(Xj,)  »  (1/6) (Xj)  (see  Fig  5-2), 

5.1.3  Gunnery  Solution  Effectiveness  Attribute  (X^) 

If  training  were  totally  effective  in  teaching  pilots  how  to  fly 
the  aircraft  to  the  proper  aim  point  to  gat  bullets  on  target,  this 
attribute  would  have  a  utility  of  one.  However,  a  utility  of  aero  is 
assumed  to  be  reached  before  the  training  reaches  an  effectiveness  of 
zero.  It  is  probable  that  if  training  Is  only  25  percent  effective, 
then  the  utility  is  aero.  Again,  a  constant  neutral  risk  linear 
equation  through  these  endpoints  defines  the  utility  function  as 
U (Xj)  -  1.33 (X^)  -  0.33  as  depicted  in  Figure  5-3. 

5.1.4  Gunnery  Tactics  Effectiveness  Attribute  (X^) 

The  utility  function  chosen  for  tactics  effectiveness  follows 
the  same  logic  as  gunnery  solution  effectiveness.  A  utility  of  one 
is  assigned  if  the  training  teaches  pilot  air  combat  maneuvering 
tactics  which  are  representative  of  actual  combat  conditions.  If 
training  becomes  only  25  percent  effective,  the  utility  is  assumed 
zero.  Thus,  U(X^)  -  1.33(X^)  -  0.33  aa  shown  in  Figure  5-4. 

5.1.5  Real  Gunnery  Events  per  Year  Attribute  (Xj) 

Most  pilots  seem  willing  to  accept  OBS  as  an  alternate  form  of 
training,  but  on  the  condition  that  n  significant  amount  of  real 
encounters  are  continued.  OBS  is  seen  as  a  way  of  augmenting  train¬ 
ing,  but  not  a  subatitute  for  air  combat  maneuvering  against  other 
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pilot*  to  refine  on*'*  competitive  edge.  A*  computed  previously* 
pilots  now  average  approximately  245  real  ancountero  per  year.  A 
utility  of  one  la  assigned  to  250  real  encounters  which  decreases  to 
zero  utility  at  zero  real  encounters.  The  utility  function  depicted 
in  Figure  5-5  is  U(Xj)  ■  0.004(X5). 

5.1.6  Availability  of  Training  Attribute  (Xg) 

Availability  of  training  is  related  to  reliability  or  the 
probability  of  achieving  any  scheduled  training  seasion.  In  the  case 
of  OB8 i  only  one  aircraft  is  required  to  become  airborne  to  perform 
the  0B8  training.  For  tactical  training*  two  aircraft  must  be 
airborne  simultaneously*  and  a  target  must  be  added  with  its  associ¬ 
ated  probability  for  live-fire  training.  If  100  percent  probability 
of  achievement  were  possible*  this  would  have  a  utility  of  one.  Zero 
utility  may  be  realised  if  this  probability  ware  to  sink  to  0.5.  The 
resulting  utility  function  shown  in  Figure  5-6  is  U(Xfi)  ■  2(X^)  -  1.0 
A  summary  of  attribute  characteristics  is  presented  in  Table  5-2. 
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U(X6)-2.0(X6)-1.0  for  X6>0.5 

■0.0  for  X,<  0.5 
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Table  5-2 


5.2  Individual  Utility  Factor  Valuaa 

Now  that  tha  tight  alternatives  and  alx  attributes  with  utility 
functions  have  been  identified,  only  one  task  remains  before  the 
utility  analyais  can  proceed.  The  remaining  task  is  to  assign  a 
specific  attribute  value  of  all  six  attributes  to  each  alternative. 
Thla  will  create  a  total  of  48  attribute  values  from  which  utility 
factors  are  determined. 

5.2.1  Utility  Factors 

To  begin,  a  cost  per  encounter  must  be  assigned  to  each  of  the 
eight  alternatives.  Goat  was  determined  previously  and  ahovn  in 
Table  5-1.  Each  coat  value  is  used  as  an  independent  variable  in  the 
exponential  utility  function,  U(X^)  ■  2.036-e0,000355^Xl\  and  solved 
for  U(X1).  Results  are  ahovn  in  Table  5-3. 

5.2.2  X2  Utility  Factors 

The  number  of  live-fire  missions,  attribute  X2,  was  also  com¬ 
puted  for  each  alternative  in  Table  5-1.  Utility  function  U(X2)  - 
<1/A)X2  is  used  to  compute  the  utility  U(X2)  for  each  alternative. 

One  correction  factor  was  applied  to  the  number  of  live-fire  missions. 
Currant  training  methods  (alternative  Al)  suffer  from  a  lack  of 
knowledge  of  miae  distance  for  thoae  firing  bursts  when  no  hits  are 
seen  hitting  the  Dart  target.  Onboard  scoring  would  eliminate  this 
deficiency  by  letting  the  pilot  know  where  his  bullet  stream  should 
have  gone  relative  to  the  target.  If  hits  are  predicted  but  the  Dart 
remains  untouched,  a  gun  alignment  problem  should  be  suspected. 

Since  alternative  Al  was  the  only  alternative  without  scoring,  credit 
was  given  for  only  one-half  of  the  live-fire  missions,  that  is,  2.1 
missions  Instead  of  4,2. 
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Table  5-3 


Utility  Factor*  for  Xj 


Alternative 

Coat  per  Encounter  (X^) 

Utility  U(X^ 

A1 

1354.27 

0.42 

A2 

1356.17 

0.42 

A3 

978.35 

0.62 

A4 

787.30 

0.71 

A5 

396.05 

0.80 

A6 

443.03 

0.87 

A7 

377.48 

0.89 

A8 

213.55 

0.96 

Table  3-4 

Utility  Factors  for  X2 


Alternative 

No.  of  Llve-Flre  Mission*  (X,,) 

mmmm 

A1 

2.7 

0.35 

A2 

4.2 

0.70 

A3 

3.9 

0.63 

A4 

3.6 

0.60 

A5 

3.2 

0.33 

A6 

2.6 

0.43 

A7 

2,1 

0.35 

A8 


0 


0.00 


b.2.3  X3  Utility  Factors 

Assessment  of  gunnery  solution  effectiveness  requires  subjective 
analysis  of  several  factors.  Remember  that  (1)  tactical  training 
encounters  are  restricted  to  less  than  135  degrees  aspect  angle  fov 
safety  purposes ,  (2)  current  LCOS  gunslghta  in  the  F-15  require  lead 
angle  at  target  aspect  angles  greater  than  40  degrees,  and  (3) 
expected  gun  opportunities  are  54  percent  front  quarter,  23  percent 
beam  quarter  (side),  and  23  percent  rear  quarter  (19).  Current 
training  alternative  (Al)  in  cactical  mode  vith  manned  target  air¬ 
craft  uses  plpper  on  target  as  criteria  for  scoring  a  kill  (19). 

Since  this  technique  is  only  correct  in  the  rear  quarter,  the  train¬ 
ing  is  only  effective  for  use  in  combat  for  the  rear  quarter  where  23 
percent  of  the  opportunities  exist.  Some  slight  additional  credit  is 
given  due  to  bullet  dispersion  and  tracking  errors  which  will  some¬ 
times  help  with  getting  hits  on  target.  An  effectiveness  rating  of 
0.3  is  assigned  to  Al. 

The  addition  of  onboard  scoring  in  alternative  A2  certainly 
improves  the  effectiveness  of  training  because  the  pilot  gets  real¬ 
time  feedback  of  where  his  bullet  stream  went  relative  to  the  target. 
BATR  on  target  can  replace  plpper  on  target  as  scoring  criteria  dur¬ 
ing  training.  Scoring  according  to  where  the  bullets  went  is  much 
more  realistic  than  plpper  on  target.  The  safety  restriction  of  135 
degree  aspect  angle  still  restricts  training  where  50  percent  of  gun 
opportunities  exist.  An  effectiveness  rating  of  0.5  is  assigned  to  A2, 

Alternatives  A3  through  A8  include  increasing  amounts  of  OBS 
training  which  does  permit  testing  at  all  aspect  angles,  including  the 
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front  quarter,  because  there  le  no  risk  of  collision  with  the 
synthetic  target.  It  is  recommended ,  however,  that  OBS  encounters 
remain  mixed  at  various  aspect  angles  and  not  limited  only  to  front 
quarter  encounters.  This  is  because  if  the  pilot  knows  OBS  targets 
Are  always  front  quarter,  he  will  learn  the  game  and  anticipate  the 
maneuvers,  which  is  less  then  realistic.  If  a  mix  of  OBS  encounters 
is  used,  then  the  pilot  would  have  to  react  to  the  particular  situa¬ 
tion  which  la  much  more  realistic.  With  a  small  fraction  of  OBS 
encounters,  some  of  the  front  quarter  void  is  filled  in.  As  a  larger 
percentage  of  OBS  is  included,  a  balance  can  be  struck  where  enough 
encounters  are  flown  to  fill  the  front  quarter  void  and  still  main¬ 
tain  sufficient  real  encounters  to  compensate  for  lens  than  perfect 
realism  in  the  synthetic  target  model.  This  balance  is  assumed  to 
occur  at  a  ratio  of  four  OBS  encounters  for  every  one  real  encounter 
(alternative  A6).  A  peak  effectiveness  rating  of  0.9  was  assigned 
simply  because  a  training  situation  will  never  fully  represent  actual 
combat.  Even  though  OBS  can  be  used  at  all  aspect  angles,  alternative 
A8,  all  OBS  and  uo  real,  was  reduced  to  0,8  effectiveness  to  account 
for  target  model  realism.  A  summary  of  Xj  attribute  values  is  shown 
in  Table  5-5  with  resulting  utility  from  U(X^)  »  1.33(Xj)  -  0.33. 


Table  5-5 


Utility  Factors  for  X^ 


A1  tentative 

Gunnery  Solution  Effectiveness  (X^) 

Utility  0(X3) 

Al 

0.3 

0.06 

A2 

0.5 

0.33 

A3 

0.6 

0.47 

A4 

0.7 

0.60 

A5 

0.8 

0.73 

A6 

0.9 

0.87 

A7 

0.85 

0.80 

A8 

0.80 

0.73 

Table  5-6 

Utility  Factors  for  X^ 


Alternative 

Gunnery  Tactics  Effectiveness  (X^) 

Utility  U(X4) 

Al 

0.6 

0.47 

A2 

0.6 

0.47 

A3 

0.75 

0.67 

A4 

0.9 

0.87 

A5 

0.8 

0.73 

A6 

0.6 

0.60 

A7 

0.4 

0.20 

A8 

0.25 

0.00 
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5.2.4  X^  Utility  Factors 

Pilot  knowledge  of  air  combat  maneuver*  la  important  in  creating 
the  positional  advantage  needed  to  develop  a  gun  shot  opportunity 
while  denying  the  adversary  opportunities.  Here,  the  relative  posi¬ 
tioning  against  the  target,  or  tactics,  are  most  important.  Lead 
angle  requirements  of  the  gvnsight  are  not  a  direct  factor  as  for  the 
previous  gun  solution  attribute  X^»  but  again,  front  quarter  safety 
restrictions  are  a  factor.  Tactical  encounters  with  manned  maneuver¬ 
ing  targets  are  generally  better  than  OBS,  at  leas  than  135  degrees 
aspect  angle,  because  of  HUD  fleld-of-view  limitations  in  OBS.  In  the 
front  quarter  OBS  can  be  somewhat  effective  since  head-on  encounters 
with  maneuvering  can  be  performed  within  the  f iold-of-view.  Current 
training  (Al)  is  aaslgnsd  An  effectiveness  of  0.6.  Alternative  A2 
remain*  0.6  because  scoring  does  nothing  to  improve  tactics.  It  is 
assumed  that  the  beat  balance  is  achieved  with  two  OBS  encounters  for 
every  one  real  with  effectiveness  at  0.9.  Then  effectiveness  continues 
to  decline  as  more  and  more  OBS  ia  included  until  alternative  AS,  all 
OBS ,,  is  assigned  an  effectiveness  of  0.25.  Applying  utility  function 
U(X^)"1 ,33(X^)-0.33  gives  results  shown  in  Table  5-6. 

5.2.5  Xj  Utility  Factors 

Pilot  preference  for  a  training  curriculum  which  contains  a  core 
of  real  encounters  is  a  significant  factor.  This  places  OBS  into  a 
training  augmentation  role  rather  than  a  replacement  of  current 
training  methods  in  the  mind  of  the  pilot.  In  Table  4-1,  the  number 
of  real  encounters  per  pilot  per  year  was  presented  for  each  of  the 
eight  alternatives.  Utility  factors  are  computed  from  U(Xj)  - 
0.004(Xg)  as  shown  in  Table  5-7. 
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Table  5-7 


Alternative 

No.  of  Real  Encounters  (X,.) 

Utility  U(X5) 

A1 

245 

0.98 

A2 

245 

0.98 

A3 

226 

0.90 

AA 

211 

0.84 

A5 

186 

0.74 

A6 

150 

0.60 

A7 

126 

0.50 

A8 

0 

0.00 

Table  5-8 

Utility  Factors  for  Xfi 


Alternative 


Availability  (Xg) 


Utility  U(Xg) 


A1 

0.79 

0.58 

A2 

0.79 

0.58 

A3 

0.81 

0.62 

A4 

0.82 

0.64 

A5 

0.84 

0.67 

A6 

0.85 

0.70 

A7 

0.86 

0.72 

A8 

0.90 

0.80 

5.2.6  Xg  Utility  Factors 

Availability  of  training  can  ba  Improved  with  application  of  OBS 
bacauaa  no  target  aircraft  is  required.  If  the  probability  of  an 
aircraft  takeoff  la  0.9.  then  the  probability  of  two  aircraft  la  0.81. 
Although  specific  F-15  mission  reliability  data  could  not  be  found,  an 
assumed  0,9  value  la  used  since  the  relative  difference  is  Important 
and  not  the  specific  value.  Adding  OBS  to  the  F-15  would  not  degrade 
mission  reliability  because  OBS  is  primarily  software  in  the  central 
computer  which  must  be  operational  for  any  training  mission.  OBS 
training  missions  are  assigned  a  probability  of  0.9  while  tactical 
missions,  which  require  two  aircraft,  are  assigned  a  probability  of 
0.81.  Live-fire  encounters  also  require  a  Dart  tow  target  which  is 
assumed  to  erode  combined  probability  to  a  value  of  0.75.  Each 
alternative  is  assigned  a  probability  based  on  the  relative  percentage 
of  OBS,  live-fire,  and  tactical  encounters.  Availability  obviously 
Improves  as  more  OBS  is  used  until  a  maximum  0.9  is  reached  with  all 
OBS.  Utility  is  determined  from  U(Xg)  ■  2(Xg)  -  1.0  as  shown  in 
Table  5-8. 

The  utility  factors  have  now  bean  assembled  for  all  alternatives 
and  attribute  combinations.  A  complete  matrix  of  the  48  utility 
factors  is  presented  in  Table  5-9.  Decision  analysis  solutions  will 
be  performed  in  the  next  chapter. 


CHAPTER  6 


MULTIATTRIBUTE  DECISION  ANALYSIS 


6.1  Form  of  ths  Utility  Function 

The  general  form  of  the  utility  function  aaeuming  attribute 
Independence!  aa  defined  by  Keeney  and  Raiffa  (9:454)  la: 

K[U(X1 i  X2,...,  Xg)]  +  1  -  ■|“lK(K1)(U1)(X1)  +  1)  Equation  6-1 

where  the  Individual  U^X1  were  determined  In  Chapter  4.  That  la,  the 
48  valuee  of  six  attributes  versus  eight  alternatives.  The  K^  terms 
are  scaling  factors  for  U^  with  values  bstween  0  and  1.  and  K  Is 
another  scaling  constant. 

6.2  Asaesslng  the  K^  Scaling  Factors 

The  procedure  for  assessing  the  K^  scaling  factors  Is  subjective 
In  that  daclelon  maker  opinion  determines  the  relative  Importance 
between  the  attributes.  Since  no  one  Individual  can  provide  an 
undisputed  Air  Force  opinion,  the  analysis  wss  performed  In  the 
following  manner.  First,  the  analysis  Is  performed  using  the  opinion 
of  the  author,  an  aerospace  engineer,  to  determine  the  scaling 
factors.  Then  the  analysis  Is  rspeated  using  the  opinions  of  a  cross 
section  of  individuals  to  determine  the  effects  of  diverse  opinions 
upon  the  analysis.  Various  backgrounds  such  as  engineers,  managers, 
current  pilots,  and  former  pilots  were  sampled. 

To  begin,  the  attributes  must  be  ranked  in  order  1  through  6  for 
level  of  Importance.  The  first  choice  is  made  by  selecting  the  one 


attribute  which  would  be  at  It*  most  preferred  level  while  ell  othere 
ere  et  their  least  preferred  level.  The  author  selected  gunnery 
solution  effectiveness,  attribute  X3,  as  the  one  most  desirable. 

Logic  for  this  choice  was.  if  a  pilot  Is  not  trained  to  hit  a  target 
that  Is  presented  to  him.  then  nothing  else  matters.  Another  choice 
Is  made  from  the  remaining  five  attributes.  X^  gunnery  tactics  was 
selected  as  the  second  ranked  attribute.  Repeating  this  procedure 
resulted  In  the  ranking  Indicated  In  Table  6-1. 

Now  the  quantitative  values  of  the  scaling  factors  must  be 
determined.  The  attribute  ranked  first  (Xj  In  the  author's  example) 
Is  paired  with  each  remaining  attribute.  For  each  pair,  the  decision 
maker  determines  at  what  level  of  the  first  ranked  attribute  would  he 
be  Indifferent  to  the  most  desirable  level  of  the  other  attribute 
(with  all  other  attributes  at  convenient  levels).  For  example.  If 
gunnery  tactics  effectiveness  X^  woe  rated  at  Its  best  possible  level 
(1.0).  at  what  level  of  gunnery  solution  effectiveness  X3  would  you 
bs  Indifferent  between  the  pair?  The  author  selected  0.95  as  tho 
value  of  X3  where  he  would  be  indifferent  to  selecting  X^  at  Its 
highest  level. 

1.0  X^  0.95  X3 

An  Indifference  value  of  0.95  means  attribute  X^  la  almost  equally 
Important  at  Xj.  As  the  Indifference  value  decreases,  the  paired 
attribute  is  of  lesoer  relative  Importance.  Indifference  equivalents 
are  established  for  all  remaining  attributes  in  a  similar  fashion. 
Results  are  presented  in  Table  6-1. 


TABLE  6-1 


Uning  the  utility  function  for  the  moae  Important  attribute  (X^) 
the  relative  acallng  factors  can  be  determined.  Aeelgnlng  ae  the 
scaling  factor  for  X3,  each  relative  scaling  factor  K^,  1  ■  1  to  6, 

Is  calculated  as  follows: 

Let  Xi  -  the  Indifference  equivalent  for  the  1th  attribute 
Then  -  l.SSfX^)  -  0.33  (the  first  ranked  utility  function) 

Continuing  the  procedure  for  tha  ongoing  example; 

XA  -  0.95 

K4  -  1.33  (0.95)  -  0.33  -  0.93 

Relative  scaling  factors  for  all  attributes  era  presented  In  Table 

6-1. 

Finally,  each  relative  scaling  factor  must  be  multiplied  by  a 
selected  value  of  K^.  Tha  results  are  tha  new  values,  tha  actual 
scaling  constants  for  each  attribute.  Here,  tha  decision  maker  must 
make  a  difficult  selection  to  the  following  question:  for  what 
probability  P.  would  you  be  Indifferent  between  option  1  with  the 
first  priority  attribute  at  its  highest  level  and  all  other  attributes 
at  their  least  dasirabls  levels,  and  an  alternative  option  2  consist¬ 
ing  of  all  attributes  at  their  most  desirable  level  with  probability 
P  or  otherwise  at  their  worst  desirable  level?  An  illustration  of 
the  two  options,  using  author  opinions  is  depicted  In  Figure  6-1. 

The  value  of  P  will  Increase  toward  a  value  of  1.0  if  the  decision 
maker  (author)  feels  that  gunnery  effectiveness  dominates  the  other 
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attributes.  If  less  dominant,  than  P  will  approach  sero.  A  P  valua 
of  0.75  waa  aalactod  by  tha  author.  Confldanea  in  tha  aalaction  of  P 
is  not  axtramaly  high.  Therefore,  tha  analysis  will  be  repeated 
using  various  values  of  P  to  establish  tha  level  of  sensitivity. 
Substituting  P  ■  0.75  for  tha  valua  of  Kj  fully  determines  tha 
magnitude  of  the  scaling  factors  as  shown  in  Table  6-1. 

Tha  valua  of  K  is  determined  by  evaluating  tha  general  form 
utility  function  at  the  attribute's  moat  preferred  values.  From  the 
scaling  convention  used  on  the  individual  attribute  utility  functions 
UA  and  composite  utility  function  U(X^,  Xg,  ...Xg),  all  utilities 
are  equal  to  1.  This  reduces  the  general  form  utility  equation  to 

K+l  ■  (KO^)  +  1)(K(K2)  +  1)...(K(K6  +  1))  Equation  6-2 

Since  all  K1  terms  are  known,  the  value  of  K  can  be  solved  by  an 
Iterative  process.  Using  author  data,  the  value  of  K  la  found  to  be 
-.987  for  the  example, 

6,3  Expected  Utility 

All  required  Information  is  now  available  to  solve  tha  general 
utility  function  (Eqn  6-1)  for  each  of  the  eight  alternatives.  The 
preferred  alternative  should  be  the  one  with  the  highest  sxpected 
utility.  Results  for  author  data  are  shown  in  Table  6-2. 

From  the  expected  utilities,  the  highest  three  alternatives  are 
nearly  equal.  This  is  not  a  significant  problsm  because  these  alter¬ 
natives  all  contain  a  mix  of  onboard  simulation.  A  decision  to  pur¬ 
chase  0B8  as  a  training  device  would  bn  recommended  over  the  other 
options. 


Table  6-2 


Author  Expected  Utilities 


Alternative 

Expected  Utility 

Als  Currant  Training 

0.78 

A2s  Currant  Training  plus  Onboard  Scoring 

0.84 

A3:  1  OBS  for  every  2  real  encounters 

0.91 

A4:  1  0B8  for  every  1  real  encounter 

0.95 

A5:  2  OBS  for  every  1  real  encounter 

0.94 

A6:  4  OBS  for  every  1  real  encounter 

0.95 

A7s  6  OBS  for  every  1  real  encounter 

0.90 

A8:  All  OBS  encounters 

0.82 

m 

ni 

PL 


a 


K-l- 

L" 


y.- 


Current  training  methods  or  adding  only  onboard  scoring  to 
current  training  are  not  as  effective.  An  average  of  approximately 
two  OBS  encounters  for  every  real  encounter  would  be  recommended. 

This  would  equate  to  1.3  dedicated  real  flights  to  every  OBS  flight. 
Total  conversion  to  OBS  training  would  not  be  desirable. 

6. A  Sensitivity  Analysis 

Before  placing  too  much  emphasis  on  the  previous  results,  some 
further  analysis  is  warranted.  Previous  concern  was  expressed  in  the 
lack  of  confidence  In  the  selection  of  probability  P  when  determining 
the  scaling  factors  for  the  six  attributes.  Values  for  P  ranging 
from  0.0  to  1.0  could  have  been  used,  with  the  author  selecting  0.75. 
The  analysis  was  recomputed  for  P  set  at  0.2.  0.4.  0.6  and  0.8  to 
determine  effects  on  the  final  results.  Table  6-3  shows  the  various 
results. 

It  is  apparent  that  the  analysis  is  not  very  sensitive  to 
changes  in  P.  Slight  differences  appear  in  the  relative  magnitude  of 
alternatives  A1  and  A8,  but  alternatives  A4,  A5,  and  A6  maintain  the 
highest  expected  utility  factors. 

6.5  Individual  Decision  Makers 

Additional  concern  Is  raised  over  the  sensitivity  of  the  analysis 
to  preferences  of  individuals  who  may  not  agree  with  opinions  of  the 
author.  An  Air  Force  pilot  who  is  current  in  F-15  aircraft  has 
different  perspectives  which  might  result  in  significantly  different 
ranking  and  scaling  of  decision  attributes.  As  a  precaution,  several 
persons  with  differing  backgrounds  ware  Interviewed  as  decision 
makers.  The  analysis  was  repeated  using  the  scaling  factors  assigned 
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by  each  subject.  A  brief  background  of  each  subject  is  presented 
below,  followed  by  Table  6-4  through  6-9  which  present  decision 
variables  for  each  individual  subject. 

Subject  No.  1:  Air  Force  Colonel  who  Is  currently  a  three-letter 
symbol  chief  in  a  research  and  development  laboratory.  Holds  a  PhD 
in  Aerospace  Engineering.  Pilot  experience  as  a  forward  air  control¬ 
ler  and  in  C— 14 1  transport. 

Subject  No.  2:  Air  Force  Major  currently  assigned  to  the  Tactical 
Air  Command  as  a  liaison  officer.  Holds  a  Masters  degree  in  Business 
Administration.  Pilot  experience  Includes  2,000  hours  in  the  F-4 
aircraft. 

Subject  No.  3:  Air  Force  Major  currently  assigned  to  the  Director 
of  Operational  Requirements  at  Headquarters,  United  States  Air  Force. 
Holds  a  Bacholor  of  Science  degree  in  Aerospace  Engineering  and  a 
Masters  degree  in  Business  Administration.  Pilot  experience  Includes 
3,000  hours  in  F-4  and,  moat  recently,  F-15  aircraft.  Also  flew  as 
test  pilot  in  gunslght  development  program. 

Subject  No.  4:  Air  Force  Captain  currently  assigned  as  Test 
Director  of  a  research  and  development  program  Investigating  air-to- 
air  gunnery  technology.  Holds  a  Bachelor  of  Science  degree  in 
Aerospace  Engineering,  Flight  experience  includes  1,200  hours  as  an 
F-4  weapon  system  officer. 

Subject  No.  5:  Air  Force  Major  currently  assigned  to  the  Tactical 
Air  Command  as  a  liaison  officer.  Holds  a  Masters  degree  and  has 
2,900  hours  ae  the  pilot  of  an  F-4  aircraft. 
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Subject  No.  6:  Air  Force  Mejor  currently  eeelgned  to  tbe 
Tactlcel  Air  Command  as  a  liaison  officer.  Holds  a  Bachelor  of 
Science  degree  In  Electrical  Engineering  and  a  Masters  degree  In 
Business  Administration.  Flight  experience  Includes  2,000  hours  ae 
an  F-4  weapon  system  officer. 

In  summary,  the  decision  makers  sampled  In  this  study  Included  an 
engineer  (author),  an  engineering  manager  (Subject  1)',  an  F-15  pilot 
(Subject  3),  two  F-4  pilots  (Subjects  2  and  5),  and  two  F-4  weapon 
system  officers  (Subjects  4  and  6).  Results  of  the  decision  variables 
selected  by  these  subjects  are  presented  in  the  summary  chapter. 


72 


TABLE  6. A 


0.75 


TABLE  6.6 


0.35 


TABLE  6,3 


CHAPTER  7 


SUMMARY 


7.1  Review  of  Decision  Maker  Responses 

Of  Che  seven  Individual  decision  makers  sampled  in  this  study, 
each  gave  a  different  rank  order  to  the  six  attributes.  Table  7-1  la 
a  summary  of  the  responses  for  the  author  and  six  other  subjects.  A 
consolidation  of  the  results  provides  soma  apparent  trends.  For 
instance,  gun  solution  effectiveness  (attribute  X^)  was  selected  as 
the  first-ranked  attribute  by  four  of  the  subjects.  Tactics  effec¬ 
tiveness  (X^)  received  top  ranking  two  times  and  availability  (Xg) 
once.  Cost  (X^),  live  fire  experience  (X^)  and  number  of  tactical 
encounters  (Xj)  were  never  ranked  first.  The  relative  Importance  of 
the  attributes  can  be  established  by  adding  the  seven  rank  values 
assigned  to  each  attribute.  Thus,  the  most  important  attribute  is 
the  one  with  the  fewest  points,  and  the  least  Important  has  the  most 
points.  Results  are  shown  in  Table  7-1.  As  expected,  gun  solution 
effectiveness  (X3)  finished  first,  followed  in  order  by  tactics 
effectiveness  (X^) ,  availability  (Xg),  cost  (Xj)  tied  with  number  of 
tactical  encounters  (X^)  and  live-fire  experience  (X^).  These 
attribute  rankings  indicate  the  attitudes  of  ths  decision  makers 
concerning  the  moat  important  characteristics  of  air-to-air  gunnery 
training.  However,  the  final  goal  of  this  study  Is  to  determine 
which  of  the  eight  decision  alternatives  should  be  selected.  There¬ 
fore,  the  decision  analysis  is  repeated  using  the  scaling  factors 
which  result  from  each  Individual  to  determine  the  expected  utility 


of  the  alternative. 


TABLE  7-1 


7.2  Expected  Utilities 

Table  7-2  Is  a  summary  of  the  expected  utilities  of  the  eight 
alternatives  which  ware  computed  for  each  decision  maker,  A  review 
of  each  alternative  Is  presented.  Alternative  Al,  the  current  form 
of  training,  received  the  lowest  expected  utility  value  In  of  the 
seven  cases,  and  had  the  lowest  average  utility.  This  strongly 
Indicates  that  OBS  can  Improve  the  quality  of  flight  training. 

However,  OBS  is  not  without  drawbacks,  as  evidenced  by  the  consistent¬ 
ly  low  utility  values  for  alternative  AB  which  would  convert  all 
training  to  OBS.  Alternative  A8  had  the  lowest  utility  value  In  two 
of  the  seven  cases  and  average  utility  only  slightly  higher  than 
alternative  Al.  It  is  apparent  that  the  best  alternative  must  be  a 
balance  of  OBS  and  "real"  training.  Addition  of  only  onboard  scoring, 
alternative  A2,  did  show  significant  improvement  over  the  current 
form  of  training.  The  level  of  Improvement  wee  not  nearly  as  great 
as  was  obtained  with  the  full  features  of  OBS,  which  Includes  the 
synthetic  target  aircraft. 

For  six  of  the  seven  subjects,  alternatives  A4  (one  OBS  per  each 
real  encounter)  and  A6  (four  OBS  per  each  real  encounter)  are  closely 
grouped  for  the  highest  expected  utility  value.  Subject  seven 
provided  the  only  exception  to  this  trend  with  alternatives  A7  (six 
OBS  per  each  real  encounter)  and  A8  (all  OBS  encounters)  also  highly 
rsted.  In  general,  the  expected  utility  decreases  ss  the  ratio  of  OBS 
encounters  drops  below  one  to  one,  or  exceeds  four  to  one.  This 
narrows  the  selection  of  best  training  mix  to  between  one  and  four  OBS 
encounters  par  each  real  encounter. 
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TABLE  7-2 


7.3  Recommendations 


Tht  average  expected  utilitiee  across  all  seven  aubjecta  for 
alternatives  A4,  A5  and  A6  are  0.94,  0.94,  and  0.95,  respectively. 

The  fidelity  of  the  decision  analysis  is  not  fine  enovgh  to  really 
distinguish  between  these  three  to  select  an  obviously  best  alterna¬ 
tive.  Since  all  three  alternatives  Involve  a  portion  of  OBS  encount¬ 
ers,  a  recommendation  to  purchase  OBS  for  operational  training  can  be 
made  with  confidence.  Only  the  specific  balance  of  training  remains 
to  be  determined. 

A  natural  balance  of  OBS  and  real  training  may  become  obvious 
after  OBS  is  available  in  operational  aircraft.  Based  upon  the  model 
developed  in  Chapter  4,  gun  training  currently  amounts  to  slightly 
greater  than  nine  percent  of  all  F-15  flights  (10,858  gun  flights  out 
of  119,712  total  flights).  Missile  training  dominates  as  the  primary 
combat  training  mode,  and  tactical  gunnery  encounters  are  generally 
piggybacked  during  the  missile  training.  Thus,  tactical  gun  training 
is  obtained  without  dedicated  flights.  Adding  an  occasional  OBS 
flight  would  achieve  a  balance  of  training. 

OBS  has  sufficient  flexibility  that  it  could  clso  be  used  in 
spot  situations  to  maximize  training  opportunities.  If  a  pilot 
destroys  a  Dart  tow  target  by  scoring  a  direct  hit,  he  could  continue 
training  with  OBS  rather  than  return  to  base.  If  two  aircraft  ara 
flying  tactical  encounters  and  one  returns  to  base  due  to  a  mainten¬ 
ance  problem,  the  remaining  aircraft  could  perform  OBS  encounters. 
Opportunistic  usea  of  OBS  could  result  in  even  greater  benefits  than 
defined  In  this  study.  Therefore,  the  exact  ratio  of  OBS  training 
should  be  optimized  based  upon  real  world  experience. 
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If  a  training  mix  of  two  OBS  ancountars  for  each  real  encounter 
is  assumed,  a  few  interesting  facts  can  be  recalled  from  Table  5-1. 
Pilots  would  receive  398.204  training  events  compared  to  175,259 
currently;  a  127  percent  Increase.  Total  flights  would  Increase  from 
10,858  to  13,533;  a  25  percent  Increase.  More  flights  and  training 
events  are  available  for  the  same  training  budget.  The  advantages 
are  obvious. 

Future  growth  potential  was  not  considered  in  the  decision 
analysis,  but  is  another  advantage  of  OBS.  Development  of  advanced 
display  concepts  like  a  helmet-mounted-display  can  be  coupled  with 
OBS  to  provide  Improved  realism.  Incorporation  of  air-to-air  missile 
engagements  in  the  software  model  could  expand  OBS  into  a  total  air 
combat  simulation.  This  future  potential  is  an  added  bonus. 

7.4  Conclusions 

The  capability  for  OBS  to  realistically  simulate  air-to-air 
gunnery  encounters  was  successfully  demonstrated  in  the  I7FC  program. 
Direct  cost  savings  of  $6,616,650  resulted  from  using  OBS  in  software 
development  and  test  and  evaluation.  Other  payoffs  Included  Improved 
safety  of  testing,  pilot  training,  and  Involvement  of  engineers  in 
the  development  testing. 

This  study  applied  decision  analysis  to  examine  the  use  of  OBS 
in  operational  flight  training  in  the  air-to-air  gunnery  mode. 

Results  strongly  indicate  that  OBS  would  Improve  the  quality  of 
training  which  can  be  achieved  within  the  existing  budget.  Thus,  Air 
Force  Investment  in  OBS  is  highly  recommended.  The  rstlo  of  OBS  and 


real  gun  encounters  Is  not  critical  and  could  vary  from  one  to  four 
OBS  encounters  for  every  real  encounter.  The  value  and  utility  of 
onboard  simulation  are  apparent. 


APPENDIX 
TABLE  A-l 

IFFC  FLIGHT  SUMMARY 


TABLE  A-l  CONTINUED 
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